[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:40:10 dist-test-slave-z8x9 krb5kdc[31472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:40:10 dist-test-slave-z8x9 krb5kdc[31472](info): set up 2 sockets
Aug 11 18:40:10 dist-test-slave-z8x9 krb5kdc[31472](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 18:40:12.208282 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.431s user 0.007s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:40:12 dist-test-slave-z8x9 krb5kdc[31472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:40:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:40:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:40:12.428045 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:38077
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:36259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:38077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:13.068648 31494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:13.069262 31494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:13.069541 31494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:13.079926 31494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:40:13.080178 31494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:13.080329 31494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:13.080525 31494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:40:13.080644 31494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:40:13.094623 31494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:36259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:38077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:38077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:13.096727 31494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:13.098807 31494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:13.123601 31505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:13.124117 31507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:13.124684 31519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:13.126009 31494 server_base.cc:1034] running on GCE node
I20260811 18:40:13.127429 31494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:13.129848 31494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:13.131130 31494 hybrid_clock.cc:648] HybridClock initialized: now 1786473613131038 us; error 107 us; skew 500 ppm
I20260811 18:40:13.131755 31494 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:13 dist-test-slave-z8x9 krb5kdc[31472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473613, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:13.136793 31494 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:40:13.139663 31494 webserver.cc:460] Webserver started at http://127.30.185.190:32781/ using document root <none> and password file <none>
I20260811 18:40:13.140817 31494 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:13.141003 31494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:13.141523 31494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:13.146256 31494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "d8c61fa6fd3d415da771ed0ee76b44b8"
format_stamp: "Formatted at 2026-08-11 18:40:13 on dist-test-slave-z8x9"
server_key: "835b1baee31df187c401b9ce9a11dd5f"
server_key_iv: "9c705faaab3d36ae5292a6e8ee0d631a"
server_key_version: "encryptionkey@0"
I20260811 18:40:13.147265 31494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "d8c61fa6fd3d415da771ed0ee76b44b8"
format_stamp: "Formatted at 2026-08-11 18:40:13 on dist-test-slave-z8x9"
server_key: "835b1baee31df187c401b9ce9a11dd5f"
server_key_iv: "9c705faaab3d36ae5292a6e8ee0d631a"
server_key_version: "encryptionkey@0"
I20260811 18:40:13.157064 31494 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.001s sys 0.009s
I20260811 18:40:13.164598 31530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:13.166905 31494 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.006s sys 0.000s
I20260811 18:40:13.167263 31494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "d8c61fa6fd3d415da771ed0ee76b44b8"
format_stamp: "Formatted at 2026-08-11 18:40:13 on dist-test-slave-z8x9"
server_key: "835b1baee31df187c401b9ce9a11dd5f"
server_key_iv: "9c705faaab3d36ae5292a6e8ee0d631a"
server_key_version: "encryptionkey@0"
I20260811 18:40:13.167574 31494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:13.198266 31494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:13.205026 31494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:13.205653 31494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:13.258518 31494 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:38077
I20260811 18:40:13.258551 31621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:38077 every 8 connection(s)
I20260811 18:40:13.260895 31494 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:40:13.267320 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 31494
I20260811 18:40:13.267845 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:40:13.268669 31622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:13.271385 31462 external_mini_cluster.cc:1428] Setting key a9713184c937dbadee2b93e4b03bf775
I20260811 18:40:13.286896 31622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8: Bootstrap starting.
I20260811 18:40:13.293505 31622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:13.295732 31622 log.cc:824] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:13.301067 31622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8: No bootstrap required, opened a new log
Aug 11 18:40:13 dist-test-slave-z8x9 krb5kdc[31472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:13.310999 31622 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8c61fa6fd3d415da771ed0ee76b44b8" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 38077 } }
I20260811 18:40:13.311637 31622 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:13.311862 31622 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8c61fa6fd3d415da771ed0ee76b44b8, State: Initialized, Role: FOLLOWER
I20260811 18:40:13.313567 31622 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8c61fa6fd3d415da771ed0ee76b44b8" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 38077 } }
I20260811 18:40:13.314059 31622 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:40:13.314464 31622 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:40:13.314716 31622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:13.318053 31622 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8c61fa6fd3d415da771ed0ee76b44b8" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 38077 } }
I20260811 18:40:13.319370 31622 leader_election.cc:302] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8c61fa6fd3d415da771ed0ee76b44b8; no voters:
I20260811 18:40:13.320312 31622 leader_election.cc:288] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:40:13.320602 31629 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:40:13.322862 31626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:13.278200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57042 (local address 127.30.185.190:38077)
0811 18:40:13.279268 (+ 1068us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:13.279293 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:13.282281 (+ 2988us) server_negotiation.cc:407] Connection header received
0811 18:40:13.292956 (+ 10675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:13.292999 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:13.293660 (+ 661us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:13.294211 (+ 551us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:13.298149 (+ 3938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:13.301044 (+ 2895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:13.303074 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:13.304405 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:13.304708 (+ 303us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:13.310872 (+ 6164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:13.310884 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:13.310919 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:13.310990 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:13.315971 (+ 4981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:13.316858 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:13.316868 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:13.316877 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:13.317099 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:13.317660 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:13.317669 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:13.317674 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:13.318814 (+ 1140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:13.319285 (+ 471us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:13.320165 (+ 880us) server_negotiation.cc:299] Negotiation successful
0811 18:40:13.320722 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":717,"thread_start_us":418,"threads_started":1}
I20260811 18:40:13.322860 31629 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8 [term 1 LEADER]: Becoming Leader. State: Replica: d8c61fa6fd3d415da771ed0ee76b44b8, State: Running, Role: LEADER
I20260811 18:40:13.323843 31629 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8c61fa6fd3d415da771ed0ee76b44b8" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 38077 } }
I20260811 18:40:13.324539 31622 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:40:13.330156 31631 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8c61fa6fd3d415da771ed0ee76b44b8. Latest consensus state: current_term: 1 leader_uuid: "d8c61fa6fd3d415da771ed0ee76b44b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8c61fa6fd3d415da771ed0ee76b44b8" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 38077 } } }
I20260811 18:40:13.330241 31630 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8c61fa6fd3d415da771ed0ee76b44b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8c61fa6fd3d415da771ed0ee76b44b8" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 38077 } } }
I20260811 18:40:13.330693 31631 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:40:13.330693 31630 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8 [sys.catalog]: This master's current role is: LEADER
W20260811 18:40:13.352617 31649 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:40:13.352912 31649 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:40:13.353300 31641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:40:13.355978 31641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:40:13.372079 31641 catalog_manager.cc:1347] Generated new cluster ID: 97f4975b10d14a6ebb35f136cca26fe1
I20260811 18:40:13.372225 31641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:40:13.394097 31641 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:40:13.395676 31641 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:40:13.415036 31641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8c61fa6fd3d415da771ed0ee76b44b8: Generated new TSK 0
I20260811 18:40:13.416488 31641 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.185.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
I20260811 18:40:13.477366 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:38077
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:36259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:13.889074 31656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:13.889478 31656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:13.889583 31656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:13.897680 31656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:13.897802 31656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:13.897957 31656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:40:13.908500 31656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:36259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:38077
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:13.909852 31656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:13.911180 31656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:13.925661 31668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:13.925803 31666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:13.925872 31672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:13.927114 31656 server_base.cc:1034] running on GCE node
I20260811 18:40:13.927721 31656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:13.928630 31656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:13.929828 31656 hybrid_clock.cc:648] HybridClock initialized: now 1786473613929758 us; error 58 us; skew 500 ppm
I20260811 18:40:13.930109 31656 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:13 dist-test-slave-z8x9 krb5kdc[31472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473613, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:13.933454 31656 init.cc:377] Logged in from keytab as kudu/127.30.185.129@KRBTEST.COM (short username kudu)
I20260811 18:40:13.935140 31656 webserver.cc:460] Webserver started at http://127.30.185.129:39229/ using document root <none> and password file <none>
I20260811 18:40:13.935889 31656 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:13.936025 31656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:13.936365 31656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:13.939452 31656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "da3857d2a9dd4d8bae0d74ed5364b892"
format_stamp: "Formatted at 2026-08-11 18:40:13 on dist-test-slave-z8x9"
server_key: "544f9510d513e6bfecd9bc2196989997"
server_key_iv: "cfe5ab610cbb59d68e11b620bac37275"
server_key_version: "encryptionkey@0"
I20260811 18:40:13.940389 31656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "da3857d2a9dd4d8bae0d74ed5364b892"
format_stamp: "Formatted at 2026-08-11 18:40:13 on dist-test-slave-z8x9"
server_key: "544f9510d513e6bfecd9bc2196989997"
server_key_iv: "cfe5ab610cbb59d68e11b620bac37275"
server_key_version: "encryptionkey@0"
I20260811 18:40:13.945778 31656 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 18:40:13.949368 31683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:13.950701 31656 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:40:13.950903 31656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "da3857d2a9dd4d8bae0d74ed5364b892"
format_stamp: "Formatted at 2026-08-11 18:40:13 on dist-test-slave-z8x9"
server_key: "544f9510d513e6bfecd9bc2196989997"
server_key_iv: "cfe5ab610cbb59d68e11b620bac37275"
server_key_version: "encryptionkey@0"
I20260811 18:40:13.951104 31656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:13.964215 31656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:13.969111 31656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:13.969552 31656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:13.970703 31656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:40:13.972596 31656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:13.972704 31656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:13.972970 31656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:13.973196 31656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:14.017668 31656 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:41637
I20260811 18:40:14.017725 31888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:41637 every 8 connection(s)
I20260811 18:40:14.019289 31656 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:40:14.025394 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 31656
I20260811 18:40:14.025647 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:40:14.026489 31462 external_mini_cluster.cc:1428] Setting key 7e65bf3aff39cc95c6f3960bbcb2b3bd
Aug 11 18:40:14 dist-test-slave-z8x9 krb5kdc[31472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473613, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:14.045147 31895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:14.022676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:54997 (local address 127.30.185.190:38077)
0811 18:40:14.023249 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:14.023256 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:14.024280 (+ 1024us) server_negotiation.cc:407] Connection header received
0811 18:40:14.027010 (+ 2730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:14.027015 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:14.027098 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:14.027196 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:14.029326 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:14.030259 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:14.033620 (+ 3361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:14.034027 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:14.034125 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:14.039012 (+ 4887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:14.039017 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:14.039021 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:14.039053 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:14.041934 (+ 2881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:14.042728 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:14.042733 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:14.042736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:14.042866 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:14.043321 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:14.043327 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:14.043335 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:14.043609 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:14.043772 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:14.044521 (+ 749us) server_negotiation.cc:299] Negotiation successful
0811 18:40:14.044753 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":304,"threads_started":1}
I20260811 18:40:14.046319 31899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:14.023229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:38077 (local address 127.30.185.129:54997)
0811 18:40:14.024106 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:14.024153 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:14.026663 (+ 2510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:14.027460 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:14.027469 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:14.028115 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:14.029081 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:14.029110 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:14.030519 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:14.030528 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:14.033423 (+ 2895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:14.033443 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:14.034292 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:14.034302 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:14.034736 (+ 434us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:14.035628 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:14.035665 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:14.038759 (+ 3094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:14.042093 (+ 3334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:14.042103 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:14.042120 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:14.042584 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:14.042983 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:14.042988 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:14.042992 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:14.043197 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:14.043747 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:14.043755 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:14.044114 (+ 359us) client_negotiation.cc:769] Sending connection context
0811 18:40:14.044407 (+ 293us) client_negotiation.cc:240] Negotiation successful
0811 18:40:14.044973 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":375,"threads_started":1}
I20260811 18:40:14.048237 31890 heartbeater.cc:347] Connected to a master server at 127.30.185.190:38077
I20260811 18:40:14.048789 31890 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:14.050064 31890 heartbeater.cc:511] Master 127.30.185.190:38077 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.185.130@KRBTEST.COM; defaulting to no policy
I20260811 18:40:14.054492 31560 ts_manager.cc:194] Registered new tserver with Master: da3857d2a9dd4d8bae0d74ed5364b892 (127.30.185.129:41637)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.130@KRBTEST.COM" created.
I20260811 18:40:14.056815 31560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:54997
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
I20260811 18:40:14.082633 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:38077
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:36259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:14.459527 31906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:14.459834 31906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:14.460002 31906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:14.467809 31906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:14.467938 31906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:14.468237 31906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:40:14.479184 31906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:36259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:38077
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:14.480561 31906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:14.481936 31906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:14.496464 31915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:14.496557 31914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:14.496519 31918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:14.497690 31906 server_base.cc:1034] running on GCE node
I20260811 18:40:14.498313 31906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:14.499251 31906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:14.500473 31906 hybrid_clock.cc:648] HybridClock initialized: now 1786473614500421 us; error 43 us; skew 500 ppm
I20260811 18:40:14.500762 31906 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:14 dist-test-slave-z8x9 krb5kdc[31472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473614, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:14.504182 31906 init.cc:377] Logged in from keytab as kudu/127.30.185.130@KRBTEST.COM (short username kudu)
I20260811 18:40:14.505839 31906 webserver.cc:460] Webserver started at http://127.30.185.130:37991/ using document root <none> and password file <none>
I20260811 18:40:14.506534 31906 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:14.506645 31906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:14.506916 31906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:14.510272 31906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "fe19730551714d64bfc2432988399d94"
format_stamp: "Formatted at 2026-08-11 18:40:14 on dist-test-slave-z8x9"
server_key: "2310041fdbe9312a8e89fbfb58fca57a"
server_key_iv: "d8e5d5d98cc3dcc55fc87ab609e6718f"
server_key_version: "encryptionkey@0"
I20260811 18:40:14.511091 31906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "fe19730551714d64bfc2432988399d94"
format_stamp: "Formatted at 2026-08-11 18:40:14 on dist-test-slave-z8x9"
server_key: "2310041fdbe9312a8e89fbfb58fca57a"
server_key_iv: "d8e5d5d98cc3dcc55fc87ab609e6718f"
server_key_version: "encryptionkey@0"
I20260811 18:40:14.516904 31906 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 18:40:14.520943 31929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:14.522410 31906 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:40:14.522550 31906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "fe19730551714d64bfc2432988399d94"
format_stamp: "Formatted at 2026-08-11 18:40:14 on dist-test-slave-z8x9"
server_key: "2310041fdbe9312a8e89fbfb58fca57a"
server_key_iv: "d8e5d5d98cc3dcc55fc87ab609e6718f"
server_key_version: "encryptionkey@0"
I20260811 18:40:14.522732 31906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:14.543416 31906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:14.547997 31906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:14.548246 31906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:14.549324 31906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:40:14.550921 31906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:14.551034 31906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:14.551136 31906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:14.551174 31906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:14.595774 31906 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:38273
I20260811 18:40:14.595877 32111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:38273 every 8 connection(s)
I20260811 18:40:14.597280 31906 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:40:14.604926 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 31906
I20260811 18:40:14.605137 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:40:14.605623 31462 external_mini_cluster.cc:1428] Setting key 093a2e35f1c31b00a4a3d1d172d68f50
Aug 11 18:40:14 dist-test-slave-z8x9 krb5kdc[31472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473614, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:14.618474 32114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:14.600152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:49825 (local address 127.30.185.190:38077)
0811 18:40:14.600627 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:14.600632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:14.601526 (+ 894us) server_negotiation.cc:407] Connection header received
0811 18:40:14.603385 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:14.603391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:14.603493 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:14.603631 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:14.605420 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:14.606231 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:14.608029 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:14.608469 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:14.608607 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:14.612287 (+ 3680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:14.612293 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:14.612296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:14.612329 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:14.615185 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:14.615942 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:14.615954 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:14.615957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:14.616078 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:14.616539 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:14.616550 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:14.616552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:14.616846 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:14.617010 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:14.617858 (+ 848us) server_negotiation.cc:299] Negotiation successful
0811 18:40:14.618094 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":274,"threads_started":1}
I20260811 18:40:14.619573 32115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:14.600548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:38077 (local address 127.30.185.130:49825)
0811 18:40:14.601356 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:14.601401 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:14.603096 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:14.603873 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:14.603881 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:14.604439 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:14.605157 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:14.605227 (+ 70us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:14.606381 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:14.606387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:14.607883 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:14.607895 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:14.608708 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:14.608715 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:14.608960 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:14.609747 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:14.609774 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:14.612028 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:14.615332 (+ 3304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:14.615341 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:14.615359 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:14.615802 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:14.616196 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:14.616201 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:14.616205 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:14.616413 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:14.616981 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:14.616990 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:14.617481 (+ 491us) client_negotiation.cc:769] Sending connection context
0811 18:40:14.617757 (+ 276us) client_negotiation.cc:240] Negotiation successful
0811 18:40:14.618321 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":321,"threads_started":1}
I20260811 18:40:14.621292 32112 heartbeater.cc:347] Connected to a master server at 127.30.185.190:38077
I20260811 18:40:14.621615 32112 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:14.622457 32112 heartbeater.cc:511] Master 127.30.185.190:38077 requested a full tablet report, sending...
I20260811 18:40:14.624193 31560 ts_manager.cc:194] Registered new tserver with Master: fe19730551714d64bfc2432988399d94 (127.30.185.130:38273)
I20260811 18:40:14.625541 31560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:49825
WARNING: no policy specified for kudu/127.30.185.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
I20260811 18:40:14.656805 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:38077
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:36259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:15.053328 32118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:15.053715 32118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:15.053901 32118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 18:40:15.060634 31890 heartbeater.cc:503] Master 127.30.185.190:38077 was elected leader, sending a full tablet report...
W20260811 18:40:15.062021 32118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:15.062168 32118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:15.062354 32118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:40:15.073110 32118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:36259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:38077
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:15.074586 32118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:15.076068 32118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:15.091094 32135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:15.091025 32127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:15.091037 32125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:15.092377 32118 server_base.cc:1034] running on GCE node
I20260811 18:40:15.093113 32118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:15.094198 32118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:15.095496 32118 hybrid_clock.cc:648] HybridClock initialized: now 1786473615095404 us; error 51 us; skew 500 ppm
I20260811 18:40:15.095919 32118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:15 dist-test-slave-z8x9 krb5kdc[31472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473615, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:15.099606 32118 init.cc:377] Logged in from keytab as kudu/127.30.185.131@KRBTEST.COM (short username kudu)
I20260811 18:40:15.101538 32118 webserver.cc:460] Webserver started at http://127.30.185.131:37823/ using document root <none> and password file <none>
I20260811 18:40:15.102268 32118 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:15.102435 32118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:15.102774 32118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:15.106031 32118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "9b5874fd6d714e879023d015b850c289"
format_stamp: "Formatted at 2026-08-11 18:40:15 on dist-test-slave-z8x9"
server_key: "83c84f49df57d1e1fa9e36f82df6d9f0"
server_key_iv: "f6d74cdaadd6701ac7b1648c9b5b0b01"
server_key_version: "encryptionkey@0"
I20260811 18:40:15.106958 32118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "9b5874fd6d714e879023d015b850c289"
format_stamp: "Formatted at 2026-08-11 18:40:15 on dist-test-slave-z8x9"
server_key: "83c84f49df57d1e1fa9e36f82df6d9f0"
server_key_iv: "f6d74cdaadd6701ac7b1648c9b5b0b01"
server_key_version: "encryptionkey@0"
I20260811 18:40:15.112658 32118 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 18:40:15.116482 32146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:15.117919 32118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 18:40:15.118105 32118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "9b5874fd6d714e879023d015b850c289"
format_stamp: "Formatted at 2026-08-11 18:40:15 on dist-test-slave-z8x9"
server_key: "83c84f49df57d1e1fa9e36f82df6d9f0"
server_key_iv: "f6d74cdaadd6701ac7b1648c9b5b0b01"
server_key_version: "encryptionkey@0"
I20260811 18:40:15.118333 32118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:15.140866 32118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:15.145787 32118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:15.146198 32118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:15.147393 32118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:40:15.148842 32118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:15.148954 32118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:15.149106 32118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:15.149207 32118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:15.195590 32118 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:41031
I20260811 18:40:15.195709 32326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:41031 every 8 connection(s)
I20260811 18:40:15.197016 32118 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:40:15.202514 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 32118
I20260811 18:40:15.202875 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:40:15.203483 31462 external_mini_cluster.cc:1428] Setting key a9e26563f57dfbcbd0b41cd207dcf3da
Aug 11 18:40:15 dist-test-slave-z8x9 krb5kdc[31472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473615, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:15.219810 32333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:15.200163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:33353 (local address 127.30.185.190:38077)
0811 18:40:15.200685 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:15.200689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:15.201833 (+ 1144us) server_negotiation.cc:407] Connection header received
0811 18:40:15.204320 (+ 2487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:15.204326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:15.204460 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:15.204591 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:15.206183 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:15.207144 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:15.208798 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:15.209247 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:15.209378 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:15.213038 (+ 3660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:15.213043 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:15.213046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:15.213077 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:15.216186 (+ 3109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:15.216938 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:15.216945 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:15.216949 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:15.217133 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:15.217671 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:15.217676 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:15.217678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:15.218030 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:15.218150 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:15.219130 (+ 980us) server_negotiation.cc:299] Negotiation successful
0811 18:40:15.219339 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":292,"threads_started":1}
I20260811 18:40:15.221385 32335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:15.200754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:38077 (local address 127.30.185.131:33353)
0811 18:40:15.201661 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:15.201704 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:15.204040 (+ 2336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:15.204833 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:15.204841 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:15.205425 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:15.206014 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:15.206031 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:15.207300 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:15.207312 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:15.208651 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:15.208664 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:15.209477 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:15.209484 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:15.209719 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:15.210350 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:15.210371 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:15.212826 (+ 2455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:15.216337 (+ 3511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:15.216345 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:15.216361 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:15.216811 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:15.217297 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:15.217313 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:15.217316 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:15.217520 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:15.218207 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:15.218218 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:15.218673 (+ 455us) client_negotiation.cc:769] Sending connection context
0811 18:40:15.218996 (+ 323us) client_negotiation.cc:240] Negotiation successful
0811 18:40:15.219601 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":478,"threads_started":1}
I20260811 18:40:15.222812 32330 heartbeater.cc:347] Connected to a master server at 127.30.185.190:38077
I20260811 18:40:15.223145 32330 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:15.223976 32330 heartbeater.cc:511] Master 127.30.185.190:38077 requested a full tablet report, sending...
I20260811 18:40:15.225667 31560 ts_manager.cc:194] Registered new tserver with Master: 9b5874fd6d714e879023d015b850c289 (127.30.185.131:41031)
I20260811 18:40:15.226625 31560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:33353
I20260811 18:40:15.232896 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 18:40:15 dist-test-slave-z8x9 krb5kdc[31472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473615, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:40:15 dist-test-slave-z8x9 krb5kdc[31472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473615, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.185.129@KRBTEST.COM
I20260811 18:40:15.301605 32342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:15.283266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60328 (local address 127.30.185.129:41637)
0811 18:40:15.284475 (+ 1209us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:15.284485 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:15.284519 (+ 34us) server_negotiation.cc:407] Connection header received
0811 18:40:15.284595 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:15.284604 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:15.284856 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:15.284989 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:15.286620 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:15.287891 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:15.289460 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:15.290065 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:15.290177 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:15.294967 (+ 4790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:15.294983 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:15.294999 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:15.295050 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:15.298462 (+ 3412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:15.299183 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:15.299191 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:15.299193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:15.299347 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:15.299718 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:15.299723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:15.299726 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:15.300281 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:15.300592 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:15.300875 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 18:40:15.301137 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":968,"thread_start_us":778,"threads_started":1}
W20260811 18:40:15.303248 31729 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60328
Aug 11 18:40:15 dist-test-slave-z8x9 krb5kdc[31472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:40:15 dist-test-slave-z8x9 krb5kdc[31472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.129@KRBTEST.COM
I20260811 18:40:15.362210 32342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:15.346681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60336 (local address 127.30.185.129:41637)
0811 18:40:15.346949 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:15.346957 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:15.347331 (+ 374us) server_negotiation.cc:407] Connection header received
0811 18:40:15.347567 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:15.347571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:15.347637 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:15.347751 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:15.349403 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:15.350166 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:15.351611 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:15.352044 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:15.352183 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:15.356564 (+ 4381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:15.356570 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:15.356573 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:15.356607 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:15.359481 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:15.360134 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:15.360142 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:15.360146 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:15.360313 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:15.360681 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:15.360686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:15.360690 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:15.360938 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:15.361231 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:15.361581 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 18:40:15.361857 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 18:40:15.364081 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 31656
I20260811 18:40:15.377630 32342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:15.365115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60352 (local address 127.30.185.129:41637)
0811 18:40:15.365381 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:15.365387 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:15.365597 (+ 210us) server_negotiation.cc:407] Connection header received
0811 18:40:15.365708 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:15.365711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:15.365767 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:15.365869 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:15.367198 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:15.367901 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:15.369449 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:15.369881 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:15.369975 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:15.372404 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:15.372409 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:15.372413 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:15.372441 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:15.375181 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:15.375748 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:15.375752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:15.375754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:15.375880 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:15.376181 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:15.376184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:15.376186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:15.376441 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:15.376622 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:15.376981 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 18:40:15.377156 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 18:40:15.378459 31882 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:15.469394 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 31656
I20260811 18:40:15.494369 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 31906
Aug 11 18:40:15 dist-test-slave-z8x9 krb5kdc[31472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.130@KRBTEST.COM
I20260811 18:40:15.511327 32352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:15.495372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45248 (local address 127.30.185.130:38273)
0811 18:40:15.495894 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:15.495902 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:15.495927 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:40:15.495989 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:15.495996 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:15.496201 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:15.496333 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:15.497662 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:15.498813 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:15.500025 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:15.500585 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:15.500684 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:15.505284 (+ 4600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:15.505297 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:15.505313 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:15.505359 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:15.508416 (+ 3057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:15.509396 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:15.509409 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:15.509414 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:15.509553 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:15.509955 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:15.509959 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:15.509961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:15.510315 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:15.510558 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:15.510802 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 18:40:15.511002 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":247,"threads_started":1}
I20260811 18:40:15.512625 32102 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:15.596982 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 31906
I20260811 18:40:15.618822 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 32118
Aug 11 18:40:15 dist-test-slave-z8x9 krb5kdc[31472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.131@KRBTEST.COM
I20260811 18:40:15.635478 32354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:15.619815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46670 (local address 127.30.185.131:41031)
0811 18:40:15.620335 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:15.620342 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:15.620371 (+ 29us) server_negotiation.cc:407] Connection header received
0811 18:40:15.620433 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:15.620439 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:15.620646 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:15.620829 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:15.622181 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:15.623270 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:15.624613 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:15.625383 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:15.625492 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:15.629936 (+ 4444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:15.629991 (+ 55us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:15.630013 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:15.630069 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:15.632811 (+ 2742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:15.633620 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:15.633626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:15.633628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:15.633754 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:15.634080 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:15.634083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:15.634085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:15.634434 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:15.634660 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:15.634953 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 18:40:15.635160 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":241,"threads_started":1}
I20260811 18:40:15.636765 32322 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:15.721643 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 32118
I20260811 18:40:15.748461 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 31494
I20260811 18:40:15.749785 31612 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:15.855005 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 31494
2026-08-11T18:40:15Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (5155 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:40:15 dist-test-slave-z8x9 krb5kdc[32362](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:40:15 dist-test-slave-z8x9 krb5kdc[32362](info): set up 2 sockets
Aug 11 18:40:15 dist-test-slave-z8x9 krb5kdc[32362](info): commencing operation
krb5kdc: starting...
W20260811 18:40:17.253785 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.340s user 0.007s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:40:17 dist-test-slave-z8x9 krb5kdc[32362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:40:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:40:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:40:17.475062 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:32805
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:38581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:32805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:17.812644 32386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:17.813063 32386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:17.813287 32386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:17.820962 32386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:40:17.821162 32386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:17.821313 32386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:17.821415 32386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:40:17.821506 32386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:40:17.832464 32386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:32805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:32805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:17.834123 32386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:17.835592 32386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:17.849782 32394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:17.849819 32399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:17.849854 32393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:17.850656 32386 server_base.cc:1034] running on GCE node
I20260811 18:40:17.851660 32386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:17.853822 32386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:17.855089 32386 hybrid_clock.cc:648] HybridClock initialized: now 1786473617855033 us; error 53 us; skew 500 ppm
I20260811 18:40:17.855584 32386 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:17 dist-test-slave-z8x9 krb5kdc[32362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473617, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:17.859467 32386 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:40:17.861321 32386 webserver.cc:460] Webserver started at http://127.30.185.190:37233/ using document root <none> and password file <none>
I20260811 18:40:17.862104 32386 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:17.862246 32386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:17.862601 32386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:17.865913 32386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "242215dd73c54a20aa4960ac11f286a9"
format_stamp: "Formatted at 2026-08-11 18:40:17 on dist-test-slave-z8x9"
server_key: "e9bf8ea90165bad884eccadca7bf3390"
server_key_iv: "c7b9734fc5991cbb0c1670c30ab6de7f"
server_key_version: "encryptionkey@0"
I20260811 18:40:17.866751 32386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "242215dd73c54a20aa4960ac11f286a9"
format_stamp: "Formatted at 2026-08-11 18:40:17 on dist-test-slave-z8x9"
server_key: "e9bf8ea90165bad884eccadca7bf3390"
server_key_iv: "c7b9734fc5991cbb0c1670c30ab6de7f"
server_key_version: "encryptionkey@0"
I20260811 18:40:17.872833 32386 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 18:40:17.877427 32411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:17.879231 32386 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:40:17.879490 32386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "242215dd73c54a20aa4960ac11f286a9"
format_stamp: "Formatted at 2026-08-11 18:40:17 on dist-test-slave-z8x9"
server_key: "e9bf8ea90165bad884eccadca7bf3390"
server_key_iv: "c7b9734fc5991cbb0c1670c30ab6de7f"
server_key_version: "encryptionkey@0"
I20260811 18:40:17.879750 32386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:17.901837 32386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:17.906361 32386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:17.906697 32386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:17.927964 32386 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:32805
I20260811 18:40:17.927986 32496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:32805 every 8 connection(s)
I20260811 18:40:17.929535 32386 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:40:17.933457 32497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:17.937669 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 32386
I20260811 18:40:17.937959 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:40:17.938601 31462 external_mini_cluster.cc:1428] Setting key c395a4832b4f90f2aec6e0f68d9519ba
I20260811 18:40:17.945254 32497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 242215dd73c54a20aa4960ac11f286a9: Bootstrap starting.
I20260811 18:40:17.949788 32497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 242215dd73c54a20aa4960ac11f286a9: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:17.951050 32497 log.cc:824] T 00000000000000000000000000000000 P 242215dd73c54a20aa4960ac11f286a9: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:17.954298 32497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 242215dd73c54a20aa4960ac11f286a9: No bootstrap required, opened a new log
Aug 11 18:40:17 dist-test-slave-z8x9 krb5kdc[32362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:17.964665 32497 raft_consensus.cc:348] T 00000000000000000000000000000000 P 242215dd73c54a20aa4960ac11f286a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242215dd73c54a20aa4960ac11f286a9" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 32805 } }
I20260811 18:40:17.965111 32497 raft_consensus.cc:374] T 00000000000000000000000000000000 P 242215dd73c54a20aa4960ac11f286a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:17.965299 32497 raft_consensus.cc:729] T 00000000000000000000000000000000 P 242215dd73c54a20aa4960ac11f286a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242215dd73c54a20aa4960ac11f286a9, State: Initialized, Role: FOLLOWER
I20260811 18:40:17.965735 32504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:17.940702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43270 (local address 127.30.185.190:32805)
0811 18:40:17.941543 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:17.941559 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:17.941606 (+ 47us) server_negotiation.cc:407] Connection header received
0811 18:40:17.943691 (+ 2085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:17.943716 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:17.944123 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:17.944457 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:17.946550 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:17.948584 (+ 2034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:17.950459 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:17.951150 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:17.951359 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:17.956564 (+ 5205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:17.956573 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:17.956589 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:17.956638 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:17.960185 (+ 3547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:17.961314 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:17.961323 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:17.961334 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:17.961573 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:17.962069 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:17.962073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:17.962075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:17.962781 (+ 706us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:17.963038 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:17.963715 (+ 677us) server_negotiation.cc:299] Negotiation successful
0811 18:40:17.964266 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":402,"threads_started":1}
I20260811 18:40:17.966571 32497 consensus_queue.cc:260] T 00000000000000000000000000000000 P 242215dd73c54a20aa4960ac11f286a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242215dd73c54a20aa4960ac11f286a9" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 32805 } }
I20260811 18:40:17.966836 32497 raft_consensus.cc:388] T 00000000000000000000000000000000 P 242215dd73c54a20aa4960ac11f286a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:40:17.967010 32497 raft_consensus.cc:482] T 00000000000000000000000000000000 P 242215dd73c54a20aa4960ac11f286a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:40:17.967259 32497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 242215dd73c54a20aa4960ac11f286a9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:18.015870 32497 raft_consensus.cc:504] T 00000000000000000000000000000000 P 242215dd73c54a20aa4960ac11f286a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242215dd73c54a20aa4960ac11f286a9" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 32805 } }
I20260811 18:40:18.016582 32497 leader_election.cc:302] T 00000000000000000000000000000000 P 242215dd73c54a20aa4960ac11f286a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 242215dd73c54a20aa4960ac11f286a9; no voters:
I20260811 18:40:18.017446 32497 leader_election.cc:288] T 00000000000000000000000000000000 P 242215dd73c54a20aa4960ac11f286a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:40:18.017592 32506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 242215dd73c54a20aa4960ac11f286a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:40:18.018887 32506 raft_consensus.cc:686] T 00000000000000000000000000000000 P 242215dd73c54a20aa4960ac11f286a9 [term 1 LEADER]: Becoming Leader. State: Replica: 242215dd73c54a20aa4960ac11f286a9, State: Running, Role: LEADER
I20260811 18:40:18.019937 32497 sys_catalog.cc:564] T 00000000000000000000000000000000 P 242215dd73c54a20aa4960ac11f286a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:40:18.019832 32506 consensus_queue.cc:237] T 00000000000000000000000000000000 P 242215dd73c54a20aa4960ac11f286a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242215dd73c54a20aa4960ac11f286a9" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 32805 } }
I20260811 18:40:18.023449 32517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 242215dd73c54a20aa4960ac11f286a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 242215dd73c54a20aa4960ac11f286a9. Latest consensus state: current_term: 1 leader_uuid: "242215dd73c54a20aa4960ac11f286a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242215dd73c54a20aa4960ac11f286a9" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 32805 } } }
I20260811 18:40:18.023965 32517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 242215dd73c54a20aa4960ac11f286a9 [sys.catalog]: This master's current role is: LEADER
I20260811 18:40:18.024502 32509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 242215dd73c54a20aa4960ac11f286a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "242215dd73c54a20aa4960ac11f286a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242215dd73c54a20aa4960ac11f286a9" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 32805 } } }
I20260811 18:40:18.024878 32509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 242215dd73c54a20aa4960ac11f286a9 [sys.catalog]: This master's current role is: LEADER
I20260811 18:40:18.027539 32537 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:40:18.033028 32537 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:40:18.042398 32537 catalog_manager.cc:1347] Generated new cluster ID: 90a6919472f145ee99c8c14f24b4e855
I20260811 18:40:18.042605 32537 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:40:18.053041 32537 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:40:18.054546 32537 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:40:18.064913 32537 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 242215dd73c54a20aa4960ac11f286a9: Generated new TSK 0
I20260811 18:40:18.065887 32537 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.185.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
W20260811 18:40:18.979444 31462 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.30.185.129: real 0.904s user 0.004s sys 0.012s
I20260811 18:40:18.980854 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:32805
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:38581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:19.318624 32558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:19.318971 32558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:19.319149 32558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:19.326854 32558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:19.327034 32558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:19.327265 32558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:40:19.337672 32558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:32805
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:19.338963 32558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:19.340265 32558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:19.354701 32567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:19.354787 32570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:19.354740 32573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:19.355604 32558 server_base.cc:1034] running on GCE node
I20260811 18:40:19.356293 32558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:19.357470 32558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:19.358727 32558 hybrid_clock.cc:648] HybridClock initialized: now 1786473619358683 us; error 56 us; skew 500 ppm
I20260811 18:40:19.359071 32558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:19 dist-test-slave-z8x9 krb5kdc[32362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473619, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:19.362759 32558 init.cc:377] Logged in from keytab as kudu/127.30.185.129@KRBTEST.COM (short username kudu)
I20260811 18:40:19.364614 32558 webserver.cc:460] Webserver started at http://127.30.185.129:32989/ using document root <none> and password file <none>
I20260811 18:40:19.365455 32558 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:19.365613 32558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:19.365952 32558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:19.369148 32558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "073181cc8fc14c08b661b89fa320f16e"
format_stamp: "Formatted at 2026-08-11 18:40:19 on dist-test-slave-z8x9"
server_key: "ca19f9b8581defe73888ba12601fb08d"
server_key_iv: "db4c360582f5cf6d95e648925e640794"
server_key_version: "encryptionkey@0"
I20260811 18:40:19.370119 32558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "073181cc8fc14c08b661b89fa320f16e"
format_stamp: "Formatted at 2026-08-11 18:40:19 on dist-test-slave-z8x9"
server_key: "ca19f9b8581defe73888ba12601fb08d"
server_key_iv: "db4c360582f5cf6d95e648925e640794"
server_key_version: "encryptionkey@0"
I20260811 18:40:19.375762 32558 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 18:40:19.379716 32583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:19.381532 32558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:40:19.381769 32558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "073181cc8fc14c08b661b89fa320f16e"
format_stamp: "Formatted at 2026-08-11 18:40:19 on dist-test-slave-z8x9"
server_key: "ca19f9b8581defe73888ba12601fb08d"
server_key_iv: "db4c360582f5cf6d95e648925e640794"
server_key_version: "encryptionkey@0"
I20260811 18:40:19.382007 32558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:19.400385 32558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:19.405750 32558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:19.406096 32558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:19.407305 32558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:40:19.408933 32558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:19.409052 32558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:19.409260 32558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:19.409343 32558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 18:40:19.447420 32558 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:36461
I20260811 18:40:19.447458 32765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:36461 every 8 connection(s)
I20260811 18:40:19.449038 32558 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:40:19.454118 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 32558
I20260811 18:40:19.454403 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:40:19.455258 31462 external_mini_cluster.cc:1428] Setting key e033d3927237c5cd12a290384a359aa7
Aug 11 18:40:19 dist-test-slave-z8x9 krb5kdc[32362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473619, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:19.476887 301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:19.451882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:46007 (local address 127.30.185.190:32805)
0811 18:40:19.452360 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:19.452364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:19.453517 (+ 1153us) server_negotiation.cc:407] Connection header received
0811 18:40:19.456363 (+ 2846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:19.456368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:19.456452 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:19.456643 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:19.458964 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:19.459888 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:19.463689 (+ 3801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:19.464413 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:19.464532 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:19.468839 (+ 4307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:19.468848 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:19.468857 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:19.468906 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:19.472873 (+ 3967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:19.473897 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:19.473906 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:19.473913 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:19.474091 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:19.474611 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:19.474619 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:19.474626 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:19.475056 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:19.475228 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:19.476017 (+ 789us) server_negotiation.cc:299] Negotiation successful
0811 18:40:19.476304 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":223,"threads_started":1}
I20260811 18:40:19.477612 306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:19.452457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:32805 (local address 127.30.185.129:46007)
0811 18:40:19.453347 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:19.453394 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:19.456000 (+ 2606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:19.456932 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:19.456942 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:19.457808 (+ 866us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:19.458683 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:19.458711 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:19.460100 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:19.460110 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:19.463486 (+ 3376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:19.463507 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:19.464690 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:19.464700 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:19.465078 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:19.465986 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:19.466010 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:19.468537 (+ 2527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:19.473071 (+ 4534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:19.473079 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:19.473095 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:19.473696 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:19.474218 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:19.474222 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:19.474224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:19.474450 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:19.475207 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:19.475215 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:19.475586 (+ 371us) client_negotiation.cc:769] Sending connection context
0811 18:40:19.475844 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 18:40:19.476304 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":304,"threads_started":1}
I20260811 18:40:19.479512 32766 heartbeater.cc:347] Connected to a master server at 127.30.185.190:32805
WARNING: no policy specified for kudu/127.30.185.130@KRBTEST.COM; defaulting to no policy
I20260811 18:40:19.479892 32766 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:19.480687 32766 heartbeater.cc:511] Master 127.30.185.190:32805 requested a full tablet report, sending...
I20260811 18:40:19.483203 32438 ts_manager.cc:194] Registered new tserver with Master: 073181cc8fc14c08b661b89fa320f16e (127.30.185.129:36461)
I20260811 18:40:19.484913 32438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:46007
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
I20260811 18:40:19.536758 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:32805
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:38581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:19.870877 312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:19.871233 312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:19.871428 312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:19.879063 312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:19.879277 312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:19.879469 312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:40:19.890249 312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:32805
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:19.891667 312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:19.893409 312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:19.907178 320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:19.907208 317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:19.907291 318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:19.913087 312 server_base.cc:1034] running on GCE node
I20260811 18:40:19.913815 312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:19.914839 312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:19.916076 312 hybrid_clock.cc:648] HybridClock initialized: now 1786473619916037 us; error 59 us; skew 500 ppm
I20260811 18:40:19.916420 312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:19 dist-test-slave-z8x9 krb5kdc[32362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473619, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:19.920352 312 init.cc:377] Logged in from keytab as kudu/127.30.185.130@KRBTEST.COM (short username kudu)
I20260811 18:40:19.922221 312 webserver.cc:460] Webserver started at http://127.30.185.130:35525/ using document root <none> and password file <none>
I20260811 18:40:19.922930 312 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:19.923038 312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:19.923363 312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:19.926461 312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "422b5eea96c0489fa61001e4c6c9d6e7"
format_stamp: "Formatted at 2026-08-11 18:40:19 on dist-test-slave-z8x9"
server_key: "9d6d801c5525d1d4ad1ebf65764e0881"
server_key_iv: "d338b856261c235835ee70b5896b3dda"
server_key_version: "encryptionkey@0"
I20260811 18:40:19.927281 312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "422b5eea96c0489fa61001e4c6c9d6e7"
format_stamp: "Formatted at 2026-08-11 18:40:19 on dist-test-slave-z8x9"
server_key: "9d6d801c5525d1d4ad1ebf65764e0881"
server_key_iv: "d338b856261c235835ee70b5896b3dda"
server_key_version: "encryptionkey@0"
I20260811 18:40:19.932976 312 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:40:19.936581 329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:19.938072 312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:40:19.938246 312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "422b5eea96c0489fa61001e4c6c9d6e7"
format_stamp: "Formatted at 2026-08-11 18:40:19 on dist-test-slave-z8x9"
server_key: "9d6d801c5525d1d4ad1ebf65764e0881"
server_key_iv: "d338b856261c235835ee70b5896b3dda"
server_key_version: "encryptionkey@0"
I20260811 18:40:19.938457 312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:19.972390 312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:19.977371 312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:19.977761 312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:19.978822 312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:40:19.980258 312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:19.980340 312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:19.980513 312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:19.980579 312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:20.018991 312 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:40633
I20260811 18:40:20.019045 533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:40633 every 8 connection(s)
I20260811 18:40:20.020344 312 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:40:20.029093 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 312
I20260811 18:40:20.029332 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:40:20.029829 31462 external_mini_cluster.cc:1428] Setting key b747aa367f0ffbfe8734954f5c6422ab
Aug 11 18:40:20 dist-test-slave-z8x9 krb5kdc[32362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473619, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:20.041937 540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:20.022961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:40111 (local address 127.30.185.190:32805)
0811 18:40:20.023445 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:20.023455 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:20.024429 (+ 974us) server_negotiation.cc:407] Connection header received
0811 18:40:20.026457 (+ 2028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:20.026463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:20.026562 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:20.026683 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:20.028224 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.029004 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.031010 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.031492 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.031633 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:20.035667 (+ 4034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:20.035673 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:20.035677 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:20.035715 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:20.038641 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:20.039445 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:20.039456 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:20.039458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:20.039579 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:20.040055 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:20.040058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:20.040060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:20.040349 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:20.040520 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:20.041294 (+ 774us) server_negotiation.cc:299] Negotiation successful
0811 18:40:20.041552 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":385,"thread_start_us":241,"threads_started":1}
I20260811 18:40:20.043085 542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:20.023429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:32805 (local address 127.30.185.130:40111)
0811 18:40:20.024260 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:20.024306 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:20.026180 (+ 1874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:20.026935 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:20.026943 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:20.027473 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:20.028045 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:20.028062 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.029217 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.029225 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:20.030857 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:20.030871 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.031757 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.031768 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:20.032003 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:20.032734 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:20.032770 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:20.035389 (+ 2619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:20.038804 (+ 3415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:20.038816 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:20.038834 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:20.039301 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:20.039709 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:20.039714 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:20.039718 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:20.039918 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:20.040495 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:20.040504 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:20.040880 (+ 376us) client_negotiation.cc:769] Sending connection context
0811 18:40:20.041170 (+ 290us) client_negotiation.cc:240] Negotiation successful
0811 18:40:20.041802 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":401,"threads_started":1}
I20260811 18:40:20.044664 534 heartbeater.cc:347] Connected to a master server at 127.30.185.190:32805
I20260811 18:40:20.044991 534 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:20.045823 534 heartbeater.cc:511] Master 127.30.185.190:32805 requested a full tablet report, sending...
I20260811 18:40:20.047299 32438 ts_manager.cc:194] Registered new tserver with Master: 422b5eea96c0489fa61001e4c6c9d6e7 (127.30.185.130:40633)
I20260811 18:40:20.048161 32438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:40111
WARNING: no policy specified for kudu/127.30.185.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
I20260811 18:40:20.081075 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:32805
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:38581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:20.419548 547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:20.419929 547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:20.420081 547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:20.427623 547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:20.427735 547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:20.427865 547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:40:20.437615 547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:32805
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:20.438884 547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:20.440197 547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:20.454182 555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:20.454119 556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:20.454952 561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:20.455156 547 server_base.cc:1034] running on GCE node
I20260811 18:40:20.455917 547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:20.456923 547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:20.458160 547 hybrid_clock.cc:648] HybridClock initialized: now 1786473620458096 us; error 85 us; skew 500 ppm
I20260811 18:40:20.458552 547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:20 dist-test-slave-z8x9 krb5kdc[32362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473620, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:20.461941 547 init.cc:377] Logged in from keytab as kudu/127.30.185.131@KRBTEST.COM (short username kudu)
I20260811 18:40:20.463521 547 webserver.cc:460] Webserver started at http://127.30.185.131:43165/ using document root <none> and password file <none>
I20260811 18:40:20.464279 547 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:20.464457 547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:20.464826 547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:20.467983 547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "e595a5558bb2485cb4f1b2056398dced"
format_stamp: "Formatted at 2026-08-11 18:40:20 on dist-test-slave-z8x9"
server_key: "93523c8afaa3b3b8f3fc9299d3fcafc8"
server_key_iv: "6bdb63a44b1d502c275d73b97fa7f487"
server_key_version: "encryptionkey@0"
I20260811 18:40:20.468829 547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "e595a5558bb2485cb4f1b2056398dced"
format_stamp: "Formatted at 2026-08-11 18:40:20 on dist-test-slave-z8x9"
server_key: "93523c8afaa3b3b8f3fc9299d3fcafc8"
server_key_iv: "6bdb63a44b1d502c275d73b97fa7f487"
server_key_version: "encryptionkey@0"
I20260811 18:40:20.474551 547 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 18:40:20.478343 573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:20.479733 547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:40:20.479930 547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "e595a5558bb2485cb4f1b2056398dced"
format_stamp: "Formatted at 2026-08-11 18:40:20 on dist-test-slave-z8x9"
server_key: "93523c8afaa3b3b8f3fc9299d3fcafc8"
server_key_iv: "6bdb63a44b1d502c275d73b97fa7f487"
server_key_version: "encryptionkey@0"
I20260811 18:40:20.480149 547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:20.488457 32766 heartbeater.cc:503] Master 127.30.185.190:32805 was elected leader, sending a full tablet report...
I20260811 18:40:20.496803 547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:20.501585 547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:20.501935 547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:20.502949 547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:40:20.504343 547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:20.504448 547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:20.504612 547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:20.504695 547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:20.545429 547 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:43109
I20260811 18:40:20.545432 767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:43109 every 8 connection(s)
I20260811 18:40:20.546833 547 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:40:20.554219 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 547
I20260811 18:40:20.554394 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:40:20.554848 31462 external_mini_cluster.cc:1428] Setting key b97816a0d0899992d9d6b8b3f9d685e2
Aug 11 18:40:20 dist-test-slave-z8x9 krb5kdc[32362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473620, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:20.569730 770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:20.549621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:58997 (local address 127.30.185.190:32805)
0811 18:40:20.550314 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:20.550321 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:20.551410 (+ 1089us) server_negotiation.cc:407] Connection header received
0811 18:40:20.553478 (+ 2068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:20.553484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:20.553590 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:20.553712 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:20.555510 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.556501 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.558635 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.559187 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.559292 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:20.563103 (+ 3811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:20.563108 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:20.563111 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:20.563145 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:20.566328 (+ 3183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:20.566999 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:20.567003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:20.567005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:20.567176 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:20.567626 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:20.567629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:20.567631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:20.567965 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:20.568106 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:20.568957 (+ 851us) server_negotiation.cc:299] Negotiation successful
0811 18:40:20.569211 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":433,"threads_started":1}
I20260811 18:40:20.570794 774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:20.550299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:32805 (local address 127.30.185.131:58997)
0811 18:40:20.551234 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:20.551296 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:20.553152 (+ 1856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:20.553954 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:20.553962 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:20.554670 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:20.555300 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:20.555339 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.556667 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.556672 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:20.558508 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:20.558520 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.559405 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.559410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:20.559670 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:20.560436 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:20.560458 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:20.562900 (+ 2442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:20.566459 (+ 3559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:20.566466 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:20.566478 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:20.566891 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:20.567292 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:20.567295 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:20.567298 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:20.567522 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:20.568073 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:20.568079 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:20.568463 (+ 384us) client_negotiation.cc:769] Sending connection context
0811 18:40:20.568802 (+ 339us) client_negotiation.cc:240] Negotiation successful
0811 18:40:20.569459 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":397,"threads_started":1}
I20260811 18:40:20.572093 768 heartbeater.cc:347] Connected to a master server at 127.30.185.190:32805
I20260811 18:40:20.572417 768 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:20.573446 768 heartbeater.cc:511] Master 127.30.185.190:32805 requested a full tablet report, sending...
I20260811 18:40:20.575019 32438 ts_manager.cc:194] Registered new tserver with Master: e595a5558bb2485cb4f1b2056398dced (127.30.185.131:43109)
I20260811 18:40:20.575912 32438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:58997
I20260811 18:40:20.583426 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:40:20.606422 770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:20.591972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43278 (local address 127.30.185.190:32805)
0811 18:40:20.592262 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:20.592271 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:20.592776 (+ 505us) server_negotiation.cc:407] Connection header received
0811 18:40:20.592999 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:20.593003 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:20.593072 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:20.593199 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:20.595077 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.595837 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.597315 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.597785 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.597895 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:20.600265 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:20.600271 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:20.600275 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:20.600311 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:20.603837 (+ 3526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:20.604520 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:20.604524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:20.604527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:20.604665 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:20.605012 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:20.605016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:20.605018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:20.605421 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:20.605649 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:20.605904 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 18:40:20.606095 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 18:40:20.615310 32438 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43278:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:40:20.625391 32438 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:40:20.658041 789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:20.643125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49734 (local address 127.30.185.130:40633)
0811 18:40:20.644113 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:20.644137 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:20.645882 (+ 1745us) server_negotiation.cc:407] Connection header received
0811 18:40:20.646228 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:20.646239 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:20.646493 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:20.646732 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:20.649530 (+ 2798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.651553 (+ 2023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.654737 (+ 3184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.656312 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.656566 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:20.656935 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:20.657318 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 18:40:20.657500 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":788,"thread_start_us":564,"threads_started":1}
I20260811 18:40:20.658223 792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:20.643069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:40633 (local address 127.0.0.1:49734)
0811 18:40:20.645648 (+ 2579us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:20.645695 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:20.645927 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:20.647177 (+ 1250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:20.647189 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:20.647220 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:20.649288 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:20.649306 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.651756 (+ 2450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.651763 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:20.654546 (+ 2783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:20.654560 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.656632 (+ 2072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.656641 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:20.656925 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:20.656988 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 18:40:20.657217 (+ 229us) client_negotiation.cc:240] Negotiation successful
0811 18:40:20.657448 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2195,"mutex_wait_us":152,"thread_start_us":2562,"threads_started":1}
I20260811 18:40:20.659363 790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:20.643229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:36461 (local address 127.0.0.1:58124)
0811 18:40:20.645917 (+ 2688us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:20.645938 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:20.646072 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:20.646742 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:20.646753 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:20.646796 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:20.648667 (+ 1871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:20.648692 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.651230 (+ 2538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.651237 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:20.655237 (+ 4000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:20.655254 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.658480 (+ 3226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.658488 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:20.658724 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:20.658759 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 18:40:20.658855 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 18:40:20.658984 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2567,"mutex_wait_us":63,"thread_start_us":1826,"threads_started":1}
I20260811 18:40:20.659493 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:20.644179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58124 (local address 127.30.185.129:36461)
0811 18:40:20.645412 (+ 1233us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:20.645420 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:20.646050 (+ 630us) server_negotiation.cc:407] Connection header received
0811 18:40:20.646305 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:20.646312 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:20.646492 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:20.646712 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:20.648990 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.651055 (+ 2065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.655527 (+ 4472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.657545 (+ 2018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.657853 (+ 308us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:20.658054 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:20.658911 (+ 857us) server_negotiation.cc:299] Negotiation successful
0811 18:40:20.659097 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1064,"thread_start_us":455,"threads_started":1}
I20260811 18:40:20.663334 798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:20.646903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:43109 (local address 127.0.0.1:53702)
0811 18:40:20.649297 (+ 2394us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:20.649334 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:20.649472 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:20.650665 (+ 1193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:20.650670 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:20.650693 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:20.651681 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:20.651699 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.654577 (+ 2878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.654582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:20.657317 (+ 2735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:20.657332 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.660428 (+ 3096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.660436 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:20.660748 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:20.660788 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 18:40:20.660962 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 18:40:20.661477 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2200,"mutex_wait_us":20,"thread_start_us":1905,"threads_started":1}
I20260811 18:40:20.663841 797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:20.646298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53702 (local address 127.30.185.131:43109)
0811 18:40:20.647551 (+ 1253us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:20.647557 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:20.649605 (+ 2048us) server_negotiation.cc:407] Connection header received
0811 18:40:20.649717 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:20.649724 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:20.649996 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:20.650287 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:20.651902 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.654368 (+ 2466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.658641 (+ 4273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.660188 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.660396 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:20.660557 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:20.661136 (+ 579us) server_negotiation.cc:299] Negotiation successful
0811 18:40:20.661763 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1103,"thread_start_us":873,"threads_started":1}
I20260811 18:40:20.664072 424 tablet_service.cc:1467] Processing CreateTablet for tablet dfaea0d77d7d4095bb080a23430fd640 (DEFAULT_TABLE table=test-table [id=1923a2ecd3e2437cb32e50db93b4b94e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:20.666178 424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfaea0d77d7d4095bb080a23430fd640. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:20.665032 32658 tablet_service.cc:1467] Processing CreateTablet for tablet dfaea0d77d7d4095bb080a23430fd640 (DEFAULT_TABLE table=test-table [id=1923a2ecd3e2437cb32e50db93b4b94e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:20.668164 32658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfaea0d77d7d4095bb080a23430fd640. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:20.668224 661 tablet_service.cc:1467] Processing CreateTablet for tablet dfaea0d77d7d4095bb080a23430fd640 (DEFAULT_TABLE table=test-table [id=1923a2ecd3e2437cb32e50db93b4b94e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:20.670054 661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfaea0d77d7d4095bb080a23430fd640. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:20.680460 799 tablet_bootstrap.cc:492] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7: Bootstrap starting.
I20260811 18:40:20.685564 799 tablet_bootstrap.cc:654] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:20.687613 802 tablet_bootstrap.cc:492] T dfaea0d77d7d4095bb080a23430fd640 P e595a5558bb2485cb4f1b2056398dced: Bootstrap starting.
I20260811 18:40:20.687752 799 log.cc:824] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:20.690124 804 tablet_bootstrap.cc:492] T dfaea0d77d7d4095bb080a23430fd640 P 073181cc8fc14c08b661b89fa320f16e: Bootstrap starting.
I20260811 18:40:20.692436 802 tablet_bootstrap.cc:654] T dfaea0d77d7d4095bb080a23430fd640 P e595a5558bb2485cb4f1b2056398dced: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:20.692446 799 tablet_bootstrap.cc:492] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7: No bootstrap required, opened a new log
I20260811 18:40:20.692804 799 ts_tablet_manager.cc:1397] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260811 18:40:20.694207 804 tablet_bootstrap.cc:654] T dfaea0d77d7d4095bb080a23430fd640 P 073181cc8fc14c08b661b89fa320f16e: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:20.694262 802 log.cc:824] T dfaea0d77d7d4095bb080a23430fd640 P e595a5558bb2485cb4f1b2056398dced: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:20.695930 804 log.cc:824] T dfaea0d77d7d4095bb080a23430fd640 P 073181cc8fc14c08b661b89fa320f16e: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:20.698673 802 tablet_bootstrap.cc:492] T dfaea0d77d7d4095bb080a23430fd640 P e595a5558bb2485cb4f1b2056398dced: No bootstrap required, opened a new log
I20260811 18:40:20.698992 802 ts_tablet_manager.cc:1397] T dfaea0d77d7d4095bb080a23430fd640 P e595a5558bb2485cb4f1b2056398dced: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260811 18:40:20.700435 804 tablet_bootstrap.cc:492] T dfaea0d77d7d4095bb080a23430fd640 P 073181cc8fc14c08b661b89fa320f16e: No bootstrap required, opened a new log
I20260811 18:40:20.700718 804 ts_tablet_manager.cc:1397] T dfaea0d77d7d4095bb080a23430fd640 P 073181cc8fc14c08b661b89fa320f16e: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260811 18:40:20.703842 799 raft_consensus.cc:348] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "422b5eea96c0489fa61001e4c6c9d6e7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40633 } } peers { permanent_uuid: "073181cc8fc14c08b661b89fa320f16e" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 36461 } } peers { permanent_uuid: "e595a5558bb2485cb4f1b2056398dced" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 43109 } }
I20260811 18:40:20.704332 799 raft_consensus.cc:374] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:20.704485 799 raft_consensus.cc:729] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 422b5eea96c0489fa61001e4c6c9d6e7, State: Initialized, Role: FOLLOWER
I20260811 18:40:20.705133 799 consensus_queue.cc:260] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "422b5eea96c0489fa61001e4c6c9d6e7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40633 } } peers { permanent_uuid: "073181cc8fc14c08b661b89fa320f16e" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 36461 } } peers { permanent_uuid: "e595a5558bb2485cb4f1b2056398dced" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 43109 } }
I20260811 18:40:20.706733 534 heartbeater.cc:503] Master 127.30.185.190:32805 was elected leader, sending a full tablet report...
I20260811 18:40:20.707548 799 ts_tablet_manager.cc:1428] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260811 18:40:20.712033 802 raft_consensus.cc:348] T dfaea0d77d7d4095bb080a23430fd640 P e595a5558bb2485cb4f1b2056398dced [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "422b5eea96c0489fa61001e4c6c9d6e7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40633 } } peers { permanent_uuid: "073181cc8fc14c08b661b89fa320f16e" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 36461 } } peers { permanent_uuid: "e595a5558bb2485cb4f1b2056398dced" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 43109 } }
I20260811 18:40:20.712586 802 raft_consensus.cc:374] T dfaea0d77d7d4095bb080a23430fd640 P e595a5558bb2485cb4f1b2056398dced [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:20.712764 802 raft_consensus.cc:729] T dfaea0d77d7d4095bb080a23430fd640 P e595a5558bb2485cb4f1b2056398dced [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e595a5558bb2485cb4f1b2056398dced, State: Initialized, Role: FOLLOWER
I20260811 18:40:20.713728 802 consensus_queue.cc:260] T dfaea0d77d7d4095bb080a23430fd640 P e595a5558bb2485cb4f1b2056398dced [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "422b5eea96c0489fa61001e4c6c9d6e7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40633 } } peers { permanent_uuid: "073181cc8fc14c08b661b89fa320f16e" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 36461 } } peers { permanent_uuid: "e595a5558bb2485cb4f1b2056398dced" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 43109 } }
I20260811 18:40:20.713985 804 raft_consensus.cc:348] T dfaea0d77d7d4095bb080a23430fd640 P 073181cc8fc14c08b661b89fa320f16e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "422b5eea96c0489fa61001e4c6c9d6e7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40633 } } peers { permanent_uuid: "073181cc8fc14c08b661b89fa320f16e" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 36461 } } peers { permanent_uuid: "e595a5558bb2485cb4f1b2056398dced" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 43109 } }
I20260811 18:40:20.714484 804 raft_consensus.cc:374] T dfaea0d77d7d4095bb080a23430fd640 P 073181cc8fc14c08b661b89fa320f16e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:20.714665 804 raft_consensus.cc:729] T dfaea0d77d7d4095bb080a23430fd640 P 073181cc8fc14c08b661b89fa320f16e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073181cc8fc14c08b661b89fa320f16e, State: Initialized, Role: FOLLOWER
I20260811 18:40:20.715242 768 heartbeater.cc:503] Master 127.30.185.190:32805 was elected leader, sending a full tablet report...
I20260811 18:40:20.715281 804 consensus_queue.cc:260] T dfaea0d77d7d4095bb080a23430fd640 P 073181cc8fc14c08b661b89fa320f16e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "422b5eea96c0489fa61001e4c6c9d6e7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40633 } } peers { permanent_uuid: "073181cc8fc14c08b661b89fa320f16e" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 36461 } } peers { permanent_uuid: "e595a5558bb2485cb4f1b2056398dced" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 43109 } }
I20260811 18:40:20.716089 802 ts_tablet_manager.cc:1428] T dfaea0d77d7d4095bb080a23430fd640 P e595a5558bb2485cb4f1b2056398dced: Time spent starting tablet: real 0.017s user 0.013s sys 0.004s
I20260811 18:40:20.717623 804 ts_tablet_manager.cc:1428] T dfaea0d77d7d4095bb080a23430fd640 P 073181cc8fc14c08b661b89fa320f16e: Time spent starting tablet: real 0.016s user 0.013s sys 0.005s
W20260811 18:40:20.772641 535 tablet.cc:2369] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:40:20.798899 769 tablet.cc:2369] T dfaea0d77d7d4095bb080a23430fd640 P e595a5558bb2485cb4f1b2056398dced: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:40:20.904210 813 raft_consensus.cc:482] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:40:20.904713 813 raft_consensus.cc:504] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "422b5eea96c0489fa61001e4c6c9d6e7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40633 } } peers { permanent_uuid: "073181cc8fc14c08b661b89fa320f16e" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 36461 } } peers { permanent_uuid: "e595a5558bb2485cb4f1b2056398dced" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 43109 } }
I20260811 18:40:20.907033 813 leader_election.cc:288] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 073181cc8fc14c08b661b89fa320f16e (127.30.185.129:36461), e595a5558bb2485cb4f1b2056398dced (127.30.185.131:43109)
I20260811 18:40:20.916445 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:20.907651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:37957 (local address 127.30.185.129:36461)
0811 18:40:20.907907 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:20.907911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:20.908716 (+ 805us) server_negotiation.cc:407] Connection header received
0811 18:40:20.909191 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:20.909221 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:20.909306 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:20.909365 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:20.910947 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.911722 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.914256 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.915219 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.915356 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:20.915449 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:20.916001 (+ 552us) server_negotiation.cc:299] Negotiation successful
0811 18:40:20.916135 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 18:40:20.916718 820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:20.907808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:36461 (local address 127.30.185.130:37957)
0811 18:40:20.908518 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:20.908561 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:20.908758 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:20.909594 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:20.909598 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:20.909621 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:20.910764 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:20.910780 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.911878 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.911883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:20.914058 (+ 2175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:20.914071 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.915632 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.915644 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:20.915815 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:20.915857 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 18:40:20.915937 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 18:40:20.916101 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"mutex_wait_us":99,"thread_start_us":401,"threads_started":1}
I20260811 18:40:20.917002 821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:20.908612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:43109 (local address 127.30.185.130:41663)
0811 18:40:20.909577 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:20.909592 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:20.909730 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:20.910137 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:20.910140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:20.910159 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:20.910736 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:20.910746 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.912017 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.912021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:20.914046 (+ 2025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:20.914060 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.915659 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.915664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:20.915829 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:20.915875 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 18:40:20.915988 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 18:40:20.916124 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845,"mutex_wait_us":60,"thread_start_us":409,"threads_started":1}
I20260811 18:40:20.917562 797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:20.908882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:41663 (local address 127.30.185.131:43109)
0811 18:40:20.909094 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:20.909099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:20.909719 (+ 620us) server_negotiation.cc:407] Connection header received
0811 18:40:20.909888 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:20.909893 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:20.910008 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:20.910115 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:20.910908 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.911869 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.914225 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.915324 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.915555 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:20.915641 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:20.916043 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 18:40:20.916166 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:40:20.918501 691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfaea0d77d7d4095bb080a23430fd640" candidate_uuid: "422b5eea96c0489fa61001e4c6c9d6e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e595a5558bb2485cb4f1b2056398dced" is_pre_election: true
I20260811 18:40:20.918467 32699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfaea0d77d7d4095bb080a23430fd640" candidate_uuid: "422b5eea96c0489fa61001e4c6c9d6e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073181cc8fc14c08b661b89fa320f16e" is_pre_election: true
I20260811 18:40:20.919229 691 raft_consensus.cc:2454] T dfaea0d77d7d4095bb080a23430fd640 P e595a5558bb2485cb4f1b2056398dced [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 422b5eea96c0489fa61001e4c6c9d6e7 in term 0.
I20260811 18:40:20.919232 32699 raft_consensus.cc:2454] T dfaea0d77d7d4095bb080a23430fd640 P 073181cc8fc14c08b661b89fa320f16e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 422b5eea96c0489fa61001e4c6c9d6e7 in term 0.
I20260811 18:40:20.920215 338 leader_election.cc:302] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073181cc8fc14c08b661b89fa320f16e, 422b5eea96c0489fa61001e4c6c9d6e7; no voters:
I20260811 18:40:20.920691 813 raft_consensus.cc:2781] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:40:20.920990 813 raft_consensus.cc:482] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:40:20.921190 813 raft_consensus.cc:3037] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:20.924129 813 raft_consensus.cc:504] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "422b5eea96c0489fa61001e4c6c9d6e7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40633 } } peers { permanent_uuid: "073181cc8fc14c08b661b89fa320f16e" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 36461 } } peers { permanent_uuid: "e595a5558bb2485cb4f1b2056398dced" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 43109 } }
I20260811 18:40:20.924890 813 leader_election.cc:288] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7 [CANDIDATE]: Term 1 election: Requested vote from peers 073181cc8fc14c08b661b89fa320f16e (127.30.185.129:36461), e595a5558bb2485cb4f1b2056398dced (127.30.185.131:43109)
I20260811 18:40:20.925544 691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfaea0d77d7d4095bb080a23430fd640" candidate_uuid: "422b5eea96c0489fa61001e4c6c9d6e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e595a5558bb2485cb4f1b2056398dced"
I20260811 18:40:20.925603 32699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfaea0d77d7d4095bb080a23430fd640" candidate_uuid: "422b5eea96c0489fa61001e4c6c9d6e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073181cc8fc14c08b661b89fa320f16e"
I20260811 18:40:20.925807 691 raft_consensus.cc:3037] T dfaea0d77d7d4095bb080a23430fd640 P e595a5558bb2485cb4f1b2056398dced [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:20.925938 32699 raft_consensus.cc:3037] T dfaea0d77d7d4095bb080a23430fd640 P 073181cc8fc14c08b661b89fa320f16e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:20.928896 32699 raft_consensus.cc:2454] T dfaea0d77d7d4095bb080a23430fd640 P 073181cc8fc14c08b661b89fa320f16e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 422b5eea96c0489fa61001e4c6c9d6e7 in term 1.
I20260811 18:40:20.929191 691 raft_consensus.cc:2454] T dfaea0d77d7d4095bb080a23430fd640 P e595a5558bb2485cb4f1b2056398dced [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 422b5eea96c0489fa61001e4c6c9d6e7 in term 1.
I20260811 18:40:20.929589 338 leader_election.cc:302] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073181cc8fc14c08b661b89fa320f16e, 422b5eea96c0489fa61001e4c6c9d6e7; no voters:
I20260811 18:40:20.929991 813 raft_consensus.cc:2781] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:40:20.930701 813 raft_consensus.cc:686] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7 [term 1 LEADER]: Becoming Leader. State: Replica: 422b5eea96c0489fa61001e4c6c9d6e7, State: Running, Role: LEADER
I20260811 18:40:20.931609 813 consensus_queue.cc:237] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "422b5eea96c0489fa61001e4c6c9d6e7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40633 } } peers { permanent_uuid: "073181cc8fc14c08b661b89fa320f16e" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 36461 } } peers { permanent_uuid: "e595a5558bb2485cb4f1b2056398dced" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 43109 } }
I20260811 18:40:20.937546 32438 catalog_manager.cc:5495] T dfaea0d77d7d4095bb080a23430fd640 P 422b5eea96c0489fa61001e4c6c9d6e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 422b5eea96c0489fa61001e4c6c9d6e7 (127.30.185.130). New cstate: current_term: 1 leader_uuid: "422b5eea96c0489fa61001e4c6c9d6e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "422b5eea96c0489fa61001e4c6c9d6e7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "073181cc8fc14c08b661b89fa320f16e" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 36461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e595a5558bb2485cb4f1b2056398dced" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 43109 } health_report { overall_health: UNKNOWN } } }
W20260811 18:40:20.952001 300 tablet.cc:2369] T dfaea0d77d7d4095bb080a23430fd640 P 073181cc8fc14c08b661b89fa320f16e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 18:40:20 dist-test-slave-z8x9 krb5kdc[32362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.129@KRBTEST.COM
I20260811 18:40:20.979849 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:20.952285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58128 (local address 127.30.185.129:36461)
0811 18:40:20.952463 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:20.952467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:20.953316 (+ 849us) server_negotiation.cc:407] Connection header received
0811 18:40:20.953566 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:20.953576 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:20.953642 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:20.953750 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:20.956203 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.957018 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.958561 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:20.959206 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:20.959355 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:20.972576 (+ 13221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:20.972597 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:20.972615 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:20.972692 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:20.976297 (+ 3605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:20.977334 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:20.977343 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:20.977345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:20.977607 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:20.978123 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:20.978127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:20.978130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:20.978534 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:20.978895 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:20.979230 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 18:40:20.979487 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 18:40:21 dist-test-slave-z8x9 krb5kdc[32362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473621, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:40:21 dist-test-slave-z8x9 krb5kdc[32362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473621, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.185.129@KRBTEST.COM
I20260811 18:40:21.043473 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:21.026969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58144 (local address 127.30.185.129:36461)
0811 18:40:21.027223 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:21.027231 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:21.027581 (+ 350us) server_negotiation.cc:407] Connection header received
0811 18:40:21.027796 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:21.027801 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:21.027878 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:21.027987 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:21.029534 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:21.030383 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:21.031605 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:21.032033 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:21.032184 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:21.036819 (+ 4635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:21.036825 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:21.036828 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:21.036863 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:21.040268 (+ 3405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:21.041133 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:21.041141 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:21.041145 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:21.041371 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:21.041847 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:21.041853 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:21.041857 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:21.042167 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:21.042477 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:21.042810 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 18:40:21.043075 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 18:40:21 dist-test-slave-z8x9 krb5kdc[32362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:40:21 dist-test-slave-z8x9 krb5kdc[32362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.129@KRBTEST.COM
I20260811 18:40:21.104416 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:21.087450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58154 (local address 127.30.185.129:36461)
0811 18:40:21.087736 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:21.087742 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:21.088040 (+ 298us) server_negotiation.cc:407] Connection header received
0811 18:40:21.088194 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:21.088198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:21.088266 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:21.088374 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:21.090325 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:21.091036 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:21.092447 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:21.093050 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:21.093151 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:21.098424 (+ 5273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:21.098430 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:21.098434 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:21.098483 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:21.101522 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:21.102263 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:21.102268 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:21.102270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:21.102489 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:21.102859 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:21.102863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:21.102865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:21.103238 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:21.103449 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:21.103824 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 18:40:21.104021 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
W20260811 18:40:21.106997 32627 tablet_service.cc:2908] Rejecting scan request for tablet dfaea0d77d7d4095bb080a23430fd640: Uninitialized: clean time has not yet been initialized
I20260811 18:40:21.110752 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 32558
I20260811 18:40:21.124353 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:21.111786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58160 (local address 127.30.185.129:36461)
0811 18:40:21.111996 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:21.112001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:21.112398 (+ 397us) server_negotiation.cc:407] Connection header received
0811 18:40:21.112540 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:21.112544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:21.112613 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:21.112690 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:21.113974 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:21.114783 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:21.115917 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:21.116393 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:21.116610 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:21.118989 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:21.118994 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:21.118997 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:21.119027 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:21.121948 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:21.122639 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:21.122642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:21.122645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:21.122757 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:21.123072 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:21.123075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:21.123077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:21.123358 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:21.123619 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:21.123893 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 18:40:21.124100 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:40:21.125036 32756 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:21.218577 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 32558
I20260811 18:40:21.248389 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 312
Aug 11 18:40:21 dist-test-slave-z8x9 krb5kdc[32362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.130@KRBTEST.COM
I20260811 18:40:21.263989 847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:21.249264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49738 (local address 127.30.185.130:40633)
0811 18:40:21.249750 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:21.249756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:21.249773 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:40:21.249825 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:21.249829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:21.249897 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:21.249990 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:21.251300 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:21.252037 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:21.253388 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:21.253797 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:21.253911 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:21.258304 (+ 4393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:21.258324 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:21.258341 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:21.258378 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:21.261422 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:21.262098 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:21.262105 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:21.262107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:21.262250 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:21.262641 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:21.262644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:21.262646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:21.262991 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:21.263219 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:21.263501 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 18:40:21.263681 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":280,"threads_started":1}
I20260811 18:40:21.264716 527 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:21.358525 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 312
W20260811 18:40:21.388458 577 connection.cc:535] server connection from 127.30.185.130:41663 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:40:21.388841 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 547
Aug 11 18:40:21 dist-test-slave-z8x9 krb5kdc[32362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.131@KRBTEST.COM
I20260811 18:40:21.405102 797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:21.389682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53716 (local address 127.30.185.131:43109)
0811 18:40:21.389892 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:21.389900 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:21.389924 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:40:21.390148 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:21.390153 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:21.390231 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:21.390344 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:21.391654 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:21.392433 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:21.393976 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:21.394380 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:21.394520 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:21.399075 (+ 4555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:21.399090 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:21.399107 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:21.399149 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:21.402330 (+ 3181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:21.403087 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:21.403094 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:21.403096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:21.403242 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:21.403684 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:21.403688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:21.403690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:21.404094 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:21.404360 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:21.404615 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 18:40:21.404805 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 18:40:21.405875 762 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:21.495783 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 547
I20260811 18:40:21.525600 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 32386
I20260811 18:40:21.526888 32485 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:21.638028 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 32386
2026-08-11T18:40:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (5781 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:40:21 dist-test-slave-z8x9 krb5kdc[855](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:40:21 dist-test-slave-z8x9 krb5kdc[855](info): set up 2 sockets
Aug 11 18:40:21 dist-test-slave-z8x9 krb5kdc[855](info): commencing operation
krb5kdc: starting...
W20260811 18:40:23.040570 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.346s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:40:23 dist-test-slave-z8x9 krb5kdc[855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:40:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:40:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:40:23.285040 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:44247
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:42117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:44247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:23.617714 872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:23.618022 872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:23.618212 872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:23.625562 872 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:40:23.625679 872 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 18:40:23.625774 872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:23.625905 872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:23.625985 872 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:40:23.626102 872 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:40:23.636206 872 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:42117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:44247
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:44247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:23.637691 872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:23.639076 872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:23.651445 880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:23.651475 882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:23.651739 884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:23.652453 872 server_base.cc:1034] running on GCE node
I20260811 18:40:23.653578 872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:23.654997 872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:23.656220 872 hybrid_clock.cc:648] HybridClock initialized: now 1786473623656160 us; error 31 us; skew 500 ppm
I20260811 18:40:23.656553 872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:23 dist-test-slave-z8x9 krb5kdc[855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473623, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:23.659945 872 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:40:23.661541 872 webserver.cc:460] Webserver started at http://127.30.185.190:33769/ using document root <none> and password file <none>
I20260811 18:40:23.662257 872 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:23.662372 872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:23.662721 872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:23.665757 872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "c84a3e8a34474f25ab1592eaae79f988"
format_stamp: "Formatted at 2026-08-11 18:40:23 on dist-test-slave-z8x9"
server_key: "07a34a6771a54cc6004809d8992f21a6"
server_key_iv: "e86301f330e41d5e194496699648c030"
server_key_version: "encryptionkey@0"
I20260811 18:40:23.666568 872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "c84a3e8a34474f25ab1592eaae79f988"
format_stamp: "Formatted at 2026-08-11 18:40:23 on dist-test-slave-z8x9"
server_key: "07a34a6771a54cc6004809d8992f21a6"
server_key_iv: "e86301f330e41d5e194496699648c030"
server_key_version: "encryptionkey@0"
I20260811 18:40:23.672607 872 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260811 18:40:23.676198 894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:23.677650 872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:40:23.677850 872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "c84a3e8a34474f25ab1592eaae79f988"
format_stamp: "Formatted at 2026-08-11 18:40:23 on dist-test-slave-z8x9"
server_key: "07a34a6771a54cc6004809d8992f21a6"
server_key_iv: "e86301f330e41d5e194496699648c030"
server_key_version: "encryptionkey@0"
I20260811 18:40:23.678076 872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:23.706547 872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:23.711115 872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:23.711412 872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:23.734515 872 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:44247
I20260811 18:40:23.734542 1002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:44247 every 8 connection(s)
I20260811 18:40:23.735963 872 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:40:23.737923 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 872
I20260811 18:40:23.738140 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:40:23.738837 31462 external_mini_cluster.cc:1428] Setting key 2d89604d5b8f66ec2a6223f2b3050b8c
I20260811 18:40:23.740643 1004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 18:40:23 dist-test-slave-z8x9 krb5kdc[855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:23.756151 1004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988: Bootstrap starting.
I20260811 18:40:23.760962 1004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:23.762466 1004 log.cc:824] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:23.765853 1007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:23.741005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39232 (local address 127.30.185.190:44247)
0811 18:40:23.741934 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:23.741951 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:23.742008 (+ 57us) server_negotiation.cc:407] Connection header received
0811 18:40:23.744345 (+ 2337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:23.744377 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:23.744979 (+ 602us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:23.745519 (+ 540us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:23.746878 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:23.749313 (+ 2435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:23.750583 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:23.751782 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:23.752019 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:23.756204 (+ 4185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:23.756216 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:23.756239 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:23.756291 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:23.760801 (+ 4510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:23.761596 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:23.761605 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:23.761613 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:23.761860 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:23.762237 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:23.762243 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:23.762246 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:23.763064 (+ 818us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:23.763367 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:23.763874 (+ 507us) server_negotiation.cc:299] Negotiation successful
0811 18:40:23.764360 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":395,"threads_started":1}
I20260811 18:40:23.766358 1004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988: No bootstrap required, opened a new log
I20260811 18:40:23.775307 1004 raft_consensus.cc:348] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c84a3e8a34474f25ab1592eaae79f988" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44247 } }
I20260811 18:40:23.775657 1004 raft_consensus.cc:374] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:23.775743 1004 raft_consensus.cc:729] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c84a3e8a34474f25ab1592eaae79f988, State: Initialized, Role: FOLLOWER
I20260811 18:40:23.776244 1004 consensus_queue.cc:260] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c84a3e8a34474f25ab1592eaae79f988" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44247 } }
I20260811 18:40:23.776470 1004 raft_consensus.cc:388] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:40:23.776575 1004 raft_consensus.cc:482] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:40:23.776712 1004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:23.779291 1004 raft_consensus.cc:504] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c84a3e8a34474f25ab1592eaae79f988" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44247 } }
I20260811 18:40:23.779767 1004 leader_election.cc:302] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c84a3e8a34474f25ab1592eaae79f988; no voters:
I20260811 18:40:23.780372 1004 leader_election.cc:288] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:40:23.780544 1011 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:40:23.781286 1011 raft_consensus.cc:686] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988 [term 1 LEADER]: Becoming Leader. State: Replica: c84a3e8a34474f25ab1592eaae79f988, State: Running, Role: LEADER
I20260811 18:40:23.781841 1011 consensus_queue.cc:237] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c84a3e8a34474f25ab1592eaae79f988" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44247 } }
I20260811 18:40:23.782706 1004 sys_catalog.cc:564] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:40:23.784893 1018 sys_catalog.cc:455] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c84a3e8a34474f25ab1592eaae79f988. Latest consensus state: current_term: 1 leader_uuid: "c84a3e8a34474f25ab1592eaae79f988" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c84a3e8a34474f25ab1592eaae79f988" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44247 } } }
I20260811 18:40:23.785473 1018 sys_catalog.cc:458] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988 [sys.catalog]: This master's current role is: LEADER
I20260811 18:40:23.786593 1013 sys_catalog.cc:455] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c84a3e8a34474f25ab1592eaae79f988" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c84a3e8a34474f25ab1592eaae79f988" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44247 } } }
I20260811 18:40:23.786957 1013 sys_catalog.cc:458] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988 [sys.catalog]: This master's current role is: LEADER
I20260811 18:40:23.788961 1027 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:40:23.793874 1027 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:40:23.803691 1027 catalog_manager.cc:1347] Generated new cluster ID: 78f440133e204c9a9e2e935bbcac2638
I20260811 18:40:23.803846 1027 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:40:23.827507 1027 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:40:23.828962 1027 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:40:23.838256 1027 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988: Generated new TSK 0
I20260811 18:40:23.839164 1027 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.185.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
I20260811 18:40:23.916077 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:44247
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:42117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:24.278154 1042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:24.278466 1042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:24.278658 1042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:24.286543 1042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:24.286675 1042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:24.286856 1042 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:40:24.287002 1042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:40:24.297516 1042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:42117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.185.190:44247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:24.298878 1042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:24.300272 1042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:24.313892 1050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:24.313882 1053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:24.313939 1051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:24.314734 1042 server_base.cc:1034] running on GCE node
I20260811 18:40:24.315433 1042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:24.316414 1042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:24.317660 1042 hybrid_clock.cc:648] HybridClock initialized: now 1786473624317595 us; error 43 us; skew 500 ppm
I20260811 18:40:24.317991 1042 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:24 dist-test-slave-z8x9 krb5kdc[855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473624, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:24.321290 1042 init.cc:377] Logged in from keytab as kudu/127.30.185.129@KRBTEST.COM (short username kudu)
I20260811 18:40:24.322757 1042 webserver.cc:460] Webserver started at http://127.30.185.129:42533/ using document root <none> and password file <none>
I20260811 18:40:24.323429 1042 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:24.323520 1042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:24.323796 1042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:24.326792 1042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "4900b3066faf472f9b7e666fe9f91d25"
format_stamp: "Formatted at 2026-08-11 18:40:24 on dist-test-slave-z8x9"
server_key: "d24ce695f0cfbfc66e923f5b10775470"
server_key_iv: "bb75fb247bcdcb0b9637a6cff0f627e5"
server_key_version: "encryptionkey@0"
I20260811 18:40:24.327683 1042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "4900b3066faf472f9b7e666fe9f91d25"
format_stamp: "Formatted at 2026-08-11 18:40:24 on dist-test-slave-z8x9"
server_key: "d24ce695f0cfbfc66e923f5b10775470"
server_key_iv: "bb75fb247bcdcb0b9637a6cff0f627e5"
server_key_version: "encryptionkey@0"
I20260811 18:40:24.333045 1042 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 18:40:24.336681 1063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:24.338258 1042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:40:24.338456 1042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "4900b3066faf472f9b7e666fe9f91d25"
format_stamp: "Formatted at 2026-08-11 18:40:24 on dist-test-slave-z8x9"
server_key: "d24ce695f0cfbfc66e923f5b10775470"
server_key_iv: "bb75fb247bcdcb0b9637a6cff0f627e5"
server_key_version: "encryptionkey@0"
I20260811 18:40:24.338716 1042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:24.365031 1042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:24.369794 1042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:24.370067 1042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:24.372939 1042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:24.373032 1042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:24.373286 1042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:24.373355 1042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 11 18:40:24 dist-test-slave-z8x9 krb5kdc[855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473624, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:24.397953 1076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:24.373678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:59293 (local address 127.30.185.190:44247)
0811 18:40:24.376119 (+ 2441us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:24.376126 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:24.376148 (+ 22us) server_negotiation.cc:407] Connection header received
0811 18:40:24.377731 (+ 1583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:24.377737 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:24.377838 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:24.377970 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:24.380215 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:24.381587 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:24.384504 (+ 2917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:24.384964 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:24.385111 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:24.390238 (+ 5127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:24.390248 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:24.390252 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:24.390304 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:24.393903 (+ 3599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:24.394691 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:24.394699 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:24.394705 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:24.394891 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:24.395539 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:24.395546 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:24.395552 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:24.395867 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:24.396100 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:24.397447 (+ 1347us) server_negotiation.cc:299] Negotiation successful
0811 18:40:24.397678 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2296,"thread_start_us":2178,"threads_started":1}
I20260811 18:40:24.400303 1077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:24.374372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:44247 (local address 127.30.185.129:59293)
0811 18:40:24.375275 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:24.375330 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:24.377447 (+ 2117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:24.378212 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:24.378220 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:24.379050 (+ 830us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:24.379987 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:24.380019 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:24.381784 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:24.381794 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:24.384344 (+ 2550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:24.384357 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:24.385262 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:24.385271 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:24.385495 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:24.386344 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:24.386376 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:24.389989 (+ 3613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:24.394058 (+ 4069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:24.394071 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:24.394094 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:24.394561 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:24.395047 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:24.395052 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:24.395056 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:24.395342 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:24.396086 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:24.396099 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:24.396804 (+ 705us) client_negotiation.cc:769] Sending connection context
0811 18:40:24.397323 (+ 519us) client_negotiation.cc:240] Negotiation successful
0811 18:40:24.398126 (+ 803us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":398,"threads_started":1}
I20260811 18:40:24.423309 1076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:24.408397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:48687 (local address 127.30.185.190:44247)
0811 18:40:24.408582 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:24.408586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:24.409386 (+ 800us) server_negotiation.cc:407] Connection header received
0811 18:40:24.409600 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:24.409605 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:24.409686 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:24.409800 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:24.410810 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:24.411523 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:24.412973 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:24.413670 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:24.413828 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:24.416435 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:24.416441 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:24.416444 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:24.416500 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:24.420243 (+ 3743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:24.421233 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:24.421239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:24.421244 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:24.421384 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:24.421777 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:24.421782 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:24.421786 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:24.422082 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:24.422263 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:24.422664 (+ 401us) server_negotiation.cc:299] Negotiation successful
0811 18:40:24.422905 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 18:40:24.423413 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:24.408511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:44247 (local address 127.30.185.129:48687)
0811 18:40:24.409162 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:24.409227 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:24.409467 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:24.409828 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:24.409833 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:24.410333 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:24.410633 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:24.410642 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:24.411666 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:24.411671 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:24.412733 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:24.412773 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:24.413899 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:24.413908 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:24.414082 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:24.414853 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:24.414887 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:24.416220 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:24.420403 (+ 4183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:24.420408 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:24.420413 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:24.421079 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:24.421511 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:24.421516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:24.421520 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:24.421659 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:24.422205 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:24.422211 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:24.422414 (+ 203us) client_negotiation.cc:769] Sending connection context
0811 18:40:24.422566 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 18:40:24.422923 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":360,"threads_started":1}
I20260811 18:40:24.431543 1042 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:34513
I20260811 18:40:24.431588 1271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:34513 every 8 connection(s)
I20260811 18:40:24.432785 1042 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:40:24.433746 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 1042
I20260811 18:40:24.433974 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:40:24.434633 31462 external_mini_cluster.cc:1428] Setting key f866ccbfdae595ec44b815713a5d7e5a
I20260811 18:40:24.436298 1272 heartbeater.cc:347] Connected to a master server at 127.30.185.190:44247
I20260811 18:40:24.436543 1272 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:24.437307 1272 heartbeater.cc:511] Master 127.30.185.190:44247 requested a full tablet report, sending...
I20260811 18:40:24.439265 927 ts_manager.cc:194] Registered new tserver with Master: 4900b3066faf472f9b7e666fe9f91d25 (127.30.185.129:34513)
I20260811 18:40:24.440739 927 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:59293
WARNING: no policy specified for kudu/127.30.185.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
I20260811 18:40:24.488444 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:44247
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:42117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:24.838743 1277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:24.839059 1277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:24.839197 1277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:24.846911 1277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:24.847043 1277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:24.847137 1277 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:40:24.847221 1277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:40:24.857198 1277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:42117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.185.190:44247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:24.858515 1277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:24.859917 1277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:24.873443 1289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:24.873483 1292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:24.873530 1290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:24.874814 1277 server_base.cc:1034] running on GCE node
I20260811 18:40:24.875559 1277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:24.876600 1277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:24.877874 1277 hybrid_clock.cc:648] HybridClock initialized: now 1786473624877808 us; error 43 us; skew 500 ppm
I20260811 18:40:24.878219 1277 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:24 dist-test-slave-z8x9 krb5kdc[855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473624, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:24.881702 1277 init.cc:377] Logged in from keytab as kudu/127.30.185.130@KRBTEST.COM (short username kudu)
I20260811 18:40:24.883455 1277 webserver.cc:460] Webserver started at http://127.30.185.130:34279/ using document root <none> and password file <none>
I20260811 18:40:24.884172 1277 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:24.884263 1277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:24.884536 1277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:24.887676 1277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "721f910c1ab04a5da8a0cfedfd0bf762"
format_stamp: "Formatted at 2026-08-11 18:40:24 on dist-test-slave-z8x9"
server_key: "8fa396e240e9486079ecf38aa6714cb2"
server_key_iv: "42cb9235ac1f3930ab9e175abb38b8da"
server_key_version: "encryptionkey@0"
I20260811 18:40:24.888513 1277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "721f910c1ab04a5da8a0cfedfd0bf762"
format_stamp: "Formatted at 2026-08-11 18:40:24 on dist-test-slave-z8x9"
server_key: "8fa396e240e9486079ecf38aa6714cb2"
server_key_iv: "42cb9235ac1f3930ab9e175abb38b8da"
server_key_version: "encryptionkey@0"
I20260811 18:40:24.894094 1277 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 18:40:24.897696 1303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:24.899091 1277 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 18:40:24.899315 1277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "721f910c1ab04a5da8a0cfedfd0bf762"
format_stamp: "Formatted at 2026-08-11 18:40:24 on dist-test-slave-z8x9"
server_key: "8fa396e240e9486079ecf38aa6714cb2"
server_key_iv: "42cb9235ac1f3930ab9e175abb38b8da"
server_key_version: "encryptionkey@0"
I20260811 18:40:24.899544 1277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:24.927692 1277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:24.932487 1277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:24.932809 1277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:24.936259 1277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:24.936381 1277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:24.936568 1277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:24.936654 1277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:40:24 dist-test-slave-z8x9 krb5kdc[855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473624, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:24.964422 1315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:24.938544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:44247 (local address 127.30.185.130:49237)
0811 18:40:24.939440 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:24.939483 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:24.941509 (+ 2026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:24.942306 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:24.942318 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:24.943134 (+ 816us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:24.944278 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:24.944322 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:24.945925 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:24.945933 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:24.948465 (+ 2532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:24.948477 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:24.949737 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:24.949747 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:24.950020 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:24.950941 (+ 921us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:24.950964 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:24.954162 (+ 3198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:24.958932 (+ 4770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:24.958939 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:24.958951 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:24.959626 (+ 675us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:24.960165 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:24.960172 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:24.960176 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:24.960421 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:24.961017 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:24.961025 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:24.961571 (+ 546us) client_negotiation.cc:769] Sending connection context
0811 18:40:24.961998 (+ 427us) client_negotiation.cc:240] Negotiation successful
0811 18:40:24.962684 (+ 686us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":389,"threads_started":1}
I20260811 18:40:24.965580 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:24.937233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:49237 (local address 127.30.185.190:44247)
0811 18:40:24.937697 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:24.937701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:24.939620 (+ 1919us) server_negotiation.cc:407] Connection header received
0811 18:40:24.941814 (+ 2194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:24.941820 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:24.941914 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:24.942051 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:24.944504 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:24.945692 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:24.948622 (+ 2930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:24.949439 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:24.949614 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:24.954422 (+ 4808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:24.954432 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:24.954440 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:24.954498 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:24.958766 (+ 4268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:24.959788 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:24.959795 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:24.959814 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:24.960000 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:24.960548 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:24.960555 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:24.960559 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:24.960853 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:24.961024 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:24.964868 (+ 3844us) server_negotiation.cc:299] Negotiation successful
0811 18:40:24.965107 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":242,"threads_started":1}
I20260811 18:40:24.988673 1424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:24.971396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:44247 (local address 127.30.185.130:51913)
0811 18:40:24.972112 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:24.972136 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:24.972316 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:24.972758 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:24.972762 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:24.973347 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:24.973890 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:24.973912 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:24.975315 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:24.975319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:24.976932 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:24.976971 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:24.978519 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:24.978525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:24.978785 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:24.979639 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:24.979674 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:24.981050 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:24.985063 (+ 4013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:24.985073 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:24.985080 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:24.985833 (+ 753us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:24.986283 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:24.986289 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:24.986292 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:24.986502 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:24.987399 (+ 897us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:24.987406 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:24.987657 (+ 251us) client_negotiation.cc:769] Sending connection context
0811 18:40:24.987827 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 18:40:24.988274 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":399,"threads_started":1}
I20260811 18:40:24.989663 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:24.971328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:51913 (local address 127.30.185.190:44247)
0811 18:40:24.971493 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:24.971497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:24.972252 (+ 755us) server_negotiation.cc:407] Connection header received
0811 18:40:24.972539 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:24.972545 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:24.972622 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:24.972729 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:24.974118 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:24.975097 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:24.977151 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:24.977976 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:24.978170 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:24.981327 (+ 3157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:24.981335 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:24.981343 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:24.981389 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:24.984915 (+ 3526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:24.985984 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:24.985993 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:24.985998 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:24.986167 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:24.986684 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:24.986690 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:24.986693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:24.987140 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:24.987321 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:24.989030 (+ 1709us) server_negotiation.cc:299] Negotiation successful
0811 18:40:24.989331 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 18:40:24.990557 1277 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:33361
I20260811 18:40:24.990600 1491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:33361 every 8 connection(s)
I20260811 18:40:24.992195 1277 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:40:24.994637 1493 heartbeater.cc:347] Connected to a master server at 127.30.185.190:44247
I20260811 18:40:24.994966 1493 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:24.995965 1493 heartbeater.cc:511] Master 127.30.185.190:44247 requested a full tablet report, sending...
I20260811 18:40:24.997787 927 ts_manager.cc:194] Registered new tserver with Master: 721f910c1ab04a5da8a0cfedfd0bf762 (127.30.185.130:33361)
I20260811 18:40:24.998646 927 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:49237
I20260811 18:40:25.002532 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 1277
I20260811 18:40:25.002730 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:40:25.003387 31462 external_mini_cluster.cc:1428] Setting key a589bcc86ac3624a53c6d9a08c5b6698
WARNING: no policy specified for kudu/127.30.185.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
I20260811 18:40:25.052871 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:44247
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:42117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:25.388542 1499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:25.388895 1499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:25.389079 1499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:25.396656 1499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:25.396842 1499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:25.397006 1499 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:40:25.397161 1499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:40:25.407757 1499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:42117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.185.190:44247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:25.409207 1499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:25.410609 1499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:25.423804 1514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:25.423902 1511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:25.423882 1518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:25.424698 1499 server_base.cc:1034] running on GCE node
I20260811 18:40:25.425408 1499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:25.426352 1499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:25.427606 1499 hybrid_clock.cc:648] HybridClock initialized: now 1786473625427547 us; error 35 us; skew 500 ppm
I20260811 18:40:25.427917 1499 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:25 dist-test-slave-z8x9 krb5kdc[855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473625, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:25.431176 1499 init.cc:377] Logged in from keytab as kudu/127.30.185.131@KRBTEST.COM (short username kudu)
I20260811 18:40:25.436025 1499 webserver.cc:460] Webserver started at http://127.30.185.131:32943/ using document root <none> and password file <none>
I20260811 18:40:25.436756 1499 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:25.436898 1499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:25.437316 1499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:25.440235 1499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "4fb98c59b7d94d94af075f92b4bdd94a"
format_stamp: "Formatted at 2026-08-11 18:40:25 on dist-test-slave-z8x9"
server_key: "14d65979789e35906ff4d9a562dec705"
server_key_iv: "50cb2607197fa8f14bd9a934beee5d47"
server_key_version: "encryptionkey@0"
I20260811 18:40:25.441084 1499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "4fb98c59b7d94d94af075f92b4bdd94a"
format_stamp: "Formatted at 2026-08-11 18:40:25 on dist-test-slave-z8x9"
server_key: "14d65979789e35906ff4d9a562dec705"
server_key_iv: "50cb2607197fa8f14bd9a934beee5d47"
server_key_version: "encryptionkey@0"
I20260811 18:40:25.443639 1272 heartbeater.cc:503] Master 127.30.185.190:44247 was elected leader, sending a full tablet report...
I20260811 18:40:25.446933 1499 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 18:40:25.450481 1534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:25.451869 1499 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:40:25.452050 1499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "4fb98c59b7d94d94af075f92b4bdd94a"
format_stamp: "Formatted at 2026-08-11 18:40:25 on dist-test-slave-z8x9"
server_key: "14d65979789e35906ff4d9a562dec705"
server_key_iv: "50cb2607197fa8f14bd9a934beee5d47"
server_key_version: "encryptionkey@0"
I20260811 18:40:25.452291 1499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:25.471567 1499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:25.475981 1499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:25.476285 1499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:25.479298 1499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:25.479437 1499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:25.479645 1499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:25.479750 1499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:40:25 dist-test-slave-z8x9 krb5kdc[855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473625, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:25.506729 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:25.479794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:46775 (local address 127.30.185.190:44247)
0811 18:40:25.480019 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:25.480027 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:25.481620 (+ 1593us) server_negotiation.cc:407] Connection header received
0811 18:40:25.484235 (+ 2615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:25.484243 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:25.484336 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:25.484455 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:25.486813 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.488020 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.491627 (+ 3607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.492087 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.492237 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:25.498304 (+ 6067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:25.498311 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:25.498315 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:25.498350 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:25.502228 (+ 3878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:25.503480 (+ 1252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:25.503489 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:25.503496 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:25.503709 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:25.504440 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:25.504447 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:25.504453 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:25.504854 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:25.505031 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:25.506095 (+ 1064us) server_negotiation.cc:299] Negotiation successful
0811 18:40:25.506338 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:40:25.507872 1547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:25.480385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:44247 (local address 127.30.185.131:46775)
0811 18:40:25.481449 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:25.481503 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:25.483879 (+ 2376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:25.484720 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:25.484728 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:25.485581 (+ 853us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:25.486607 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:25.486628 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.488255 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.488259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:25.491450 (+ 3191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:25.491465 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.492366 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.492374 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:25.492647 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:25.493746 (+ 1099us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:25.493796 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:25.497955 (+ 4159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:25.502378 (+ 4423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:25.502387 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:25.502408 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:25.503275 (+ 867us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:25.503978 (+ 703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:25.503981 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:25.503984 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:25.504268 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:25.505170 (+ 902us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:25.505199 (+ 29us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:25.505671 (+ 472us) client_negotiation.cc:769] Sending connection context
0811 18:40:25.505975 (+ 304us) client_negotiation.cc:240] Negotiation successful
0811 18:40:25.506593 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":403,"threads_started":1}
I20260811 18:40:25.537694 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:25.518772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:40541 (local address 127.30.185.190:44247)
0811 18:40:25.519047 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:25.519057 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:25.519095 (+ 38us) server_negotiation.cc:407] Connection header received
0811 18:40:25.519384 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:25.519393 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:25.519504 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:25.519659 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:25.520856 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.522319 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.524840 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.525768 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.525930 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:25.528438 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:25.528448 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:25.528457 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:25.528518 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:25.533553 (+ 5035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:25.534827 (+ 1274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:25.534831 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:25.534834 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:25.534976 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:25.535381 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:25.535386 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:25.535389 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:25.535916 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:25.536105 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:25.536735 (+ 630us) server_negotiation.cc:299] Negotiation successful
0811 18:40:25.537066 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 18:40:25.538163 1644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:25.518288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:44247 (local address 127.30.185.131:40541)
0811 18:40:25.518996 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:25.519018 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:25.519191 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:25.519677 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:25.519680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:25.520144 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:25.520680 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:25.520692 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.522516 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.522524 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:25.524532 (+ 2008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:25.524556 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.526044 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.526053 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:25.526257 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:25.527244 (+ 987us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:25.527263 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:25.528158 (+ 895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:25.533716 (+ 5558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:25.533722 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:25.533726 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:25.534627 (+ 901us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:25.535107 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:25.535114 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:25.535122 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:25.535260 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:25.536195 (+ 935us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:25.536199 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:25.536392 (+ 193us) client_negotiation.cc:769] Sending connection context
0811 18:40:25.536594 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 18:40:25.537259 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":559,"threads_started":1}
I20260811 18:40:25.555462 1499 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:40971
I20260811 18:40:25.555666 1752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:40971 every 8 connection(s)
I20260811 18:40:25.557446 1499 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:40:25.560353 1753 heartbeater.cc:347] Connected to a master server at 127.30.185.190:44247
I20260811 18:40:25.560662 1753 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:25.561542 1753 heartbeater.cc:511] Master 127.30.185.190:44247 requested a full tablet report, sending...
I20260811 18:40:25.563053 927 ts_manager.cc:194] Registered new tserver with Master: 4fb98c59b7d94d94af075f92b4bdd94a (127.30.185.131:40971)
I20260811 18:40:25.563860 927 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:46775
I20260811 18:40:25.565375 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 1499
I20260811 18:40:25.565557 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:40:25.566015 31462 external_mini_cluster.cc:1428] Setting key 3efc735352b41fba45def38f48f4ed2f
I20260811 18:40:25.569566 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 18:40:25 dist-test-slave-z8x9 krb5kdc[855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473625, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:40:25 dist-test-slave-z8x9 krb5kdc[855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473625, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:25.633565 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:25.617664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39244 (local address 127.30.185.190:44247)
0811 18:40:25.617841 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:25.617846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:25.618302 (+ 456us) server_negotiation.cc:407] Connection header received
0811 18:40:25.618554 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:25.618559 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:25.618630 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:25.618735 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:25.620291 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.621046 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.622711 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.623117 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.623263 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:25.627870 (+ 4607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:25.627876 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:25.627880 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:25.627919 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:25.630984 (+ 3065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:25.631760 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:25.631764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:25.631767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:25.631902 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:25.632238 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:25.632241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:25.632243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:25.632540 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:25.632743 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:25.633044 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 18:40:25.633308 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:40:25.638279 927 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39244:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:40:25.646924 927 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:40:25.687532 1784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:25.662070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33258 (local address 127.30.185.131:40971)
0811 18:40:25.676054 (+ 13984us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:25.676068 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:25.676115 (+ 47us) server_negotiation.cc:407] Connection header received
0811 18:40:25.676250 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:25.676262 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:25.676611 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:25.676914 (+ 303us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:25.678313 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.680455 (+ 2142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.684060 (+ 3605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.686233 (+ 2173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.686515 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:25.686727 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:25.686937 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 18:40:25.687114 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13675,"thread_start_us":13481,"threads_started":1}
I20260811 18:40:25.688014 1783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:25.661616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:40971 (local address 127.0.0.1:33258)
0811 18:40:25.662944 (+ 1328us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:25.663008 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:25.663253 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:25.676955 (+ 13702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:25.676967 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:25.677015 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:25.677832 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:25.677852 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.680605 (+ 2753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.680611 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:25.683841 (+ 3230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:25.683868 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.686446 (+ 2578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.686452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:25.686650 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:25.686713 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 18:40:25.686862 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 18:40:25.687085 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":931,"mutex_wait_us":24,"thread_start_us":944,"threads_started":1}
I20260811 18:40:25.688191 1796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:25.663776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45098 (local address 127.30.185.130:33361)
0811 18:40:25.679223 (+ 15447us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:25.679232 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:25.679260 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:40:25.679358 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:25.679365 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:25.679586 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:25.679789 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:25.680763 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.682212 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.685382 (+ 3170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.686336 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.686508 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:25.686704 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:25.687602 (+ 898us) server_negotiation.cc:299] Negotiation successful
0811 18:40:25.687800 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15226,"thread_start_us":16067,"threads_started":1}
I20260811 18:40:25.688072 1791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:25.662396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:34513 (local address 127.0.0.1:53532)
0811 18:40:25.674340 (+ 11944us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:25.674368 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:25.674664 (+ 296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:25.678417 (+ 3753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:25.678422 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:25.678445 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:25.679082 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:25.679092 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.681083 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.681089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:25.684417 (+ 3328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:25.684433 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.687000 (+ 2567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.687010 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:25.687182 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:25.687224 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 18:40:25.687342 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 18:40:25.687484 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11749,"mutex_wait_us":26,"thread_start_us":11575,"threads_started":1}
I20260811 18:40:25.688437 1794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:25.663467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:33361 (local address 127.0.0.1:45098)
0811 18:40:25.675458 (+ 11991us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:25.675479 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:25.675626 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:25.679806 (+ 4180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:25.679812 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:25.679837 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:25.680536 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:25.680549 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.682348 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.682353 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:25.685140 (+ 2787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:25.685162 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.686884 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.686890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:25.687075 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:25.687102 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 18:40:25.687197 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 18:40:25.687334 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11802,"mutex_wait_us":1,"thread_start_us":11602,"threads_started":1}
I20260811 18:40:25.688799 1792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:25.662819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53532 (local address 127.30.185.129:34513)
0811 18:40:25.677621 (+ 14802us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:25.677632 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:25.677661 (+ 29us) server_negotiation.cc:407] Connection header received
0811 18:40:25.677761 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:25.677768 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:25.677984 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:25.678190 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:25.679272 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.680887 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.684646 (+ 3759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.686507 (+ 1861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.686681 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:25.686836 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:25.688239 (+ 1403us) server_negotiation.cc:299] Negotiation successful
0811 18:40:25.688437 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14625,"thread_start_us":16395,"threads_started":1}
I20260811 18:40:25.692905 1385 tablet_service.cc:1467] Processing CreateTablet for tablet b037293d5b60471082da209381a3e9bf (DEFAULT_TABLE table=test-table [id=787e9cbe53834b05816bc4dbf258d46a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:25.692905 1157 tablet_service.cc:1467] Processing CreateTablet for tablet b037293d5b60471082da209381a3e9bf (DEFAULT_TABLE table=test-table [id=787e9cbe53834b05816bc4dbf258d46a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:25.694481 1157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b037293d5b60471082da209381a3e9bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:25.694581 1385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b037293d5b60471082da209381a3e9bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:25.695537 1625 tablet_service.cc:1467] Processing CreateTablet for tablet b037293d5b60471082da209381a3e9bf (DEFAULT_TABLE table=test-table [id=787e9cbe53834b05816bc4dbf258d46a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:25.696694 1625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b037293d5b60471082da209381a3e9bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:25.711272 1798 tablet_bootstrap.cc:492] T b037293d5b60471082da209381a3e9bf P 4fb98c59b7d94d94af075f92b4bdd94a: Bootstrap starting.
I20260811 18:40:25.712754 1799 tablet_bootstrap.cc:492] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762: Bootstrap starting.
I20260811 18:40:25.716226 1798 tablet_bootstrap.cc:654] T b037293d5b60471082da209381a3e9bf P 4fb98c59b7d94d94af075f92b4bdd94a: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:25.717291 1799 tablet_bootstrap.cc:654] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:25.718062 1798 log.cc:824] T b037293d5b60471082da209381a3e9bf P 4fb98c59b7d94d94af075f92b4bdd94a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:25.718837 1800 tablet_bootstrap.cc:492] T b037293d5b60471082da209381a3e9bf P 4900b3066faf472f9b7e666fe9f91d25: Bootstrap starting.
I20260811 18:40:25.718890 1799 log.cc:824] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:25.722306 1798 tablet_bootstrap.cc:492] T b037293d5b60471082da209381a3e9bf P 4fb98c59b7d94d94af075f92b4bdd94a: No bootstrap required, opened a new log
I20260811 18:40:25.722694 1798 ts_tablet_manager.cc:1397] T b037293d5b60471082da209381a3e9bf P 4fb98c59b7d94d94af075f92b4bdd94a: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.004s
I20260811 18:40:25.722750 1799 tablet_bootstrap.cc:492] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762: No bootstrap required, opened a new log
I20260811 18:40:25.723014 1799 ts_tablet_manager.cc:1397] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 18:40:25.723186 1800 tablet_bootstrap.cc:654] T b037293d5b60471082da209381a3e9bf P 4900b3066faf472f9b7e666fe9f91d25: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:25.724886 1800 log.cc:824] T b037293d5b60471082da209381a3e9bf P 4900b3066faf472f9b7e666fe9f91d25: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:25.728732 1800 tablet_bootstrap.cc:492] T b037293d5b60471082da209381a3e9bf P 4900b3066faf472f9b7e666fe9f91d25: No bootstrap required, opened a new log
I20260811 18:40:25.729036 1800 ts_tablet_manager.cc:1397] T b037293d5b60471082da209381a3e9bf P 4900b3066faf472f9b7e666fe9f91d25: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 18:40:25.732764 1798 raft_consensus.cc:348] T b037293d5b60471082da209381a3e9bf P 4fb98c59b7d94d94af075f92b4bdd94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } }
I20260811 18:40:25.733301 1798 raft_consensus.cc:374] T b037293d5b60471082da209381a3e9bf P 4fb98c59b7d94d94af075f92b4bdd94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:25.733579 1798 raft_consensus.cc:729] T b037293d5b60471082da209381a3e9bf P 4fb98c59b7d94d94af075f92b4bdd94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fb98c59b7d94d94af075f92b4bdd94a, State: Initialized, Role: FOLLOWER
I20260811 18:40:25.734334 1798 consensus_queue.cc:260] T b037293d5b60471082da209381a3e9bf P 4fb98c59b7d94d94af075f92b4bdd94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } }
I20260811 18:40:25.735311 1799 raft_consensus.cc:348] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } }
I20260811 18:40:25.735814 1799 raft_consensus.cc:374] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:25.735970 1799 raft_consensus.cc:729] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 721f910c1ab04a5da8a0cfedfd0bf762, State: Initialized, Role: FOLLOWER
I20260811 18:40:25.736752 1799 consensus_queue.cc:260] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } }
I20260811 18:40:25.743805 1800 raft_consensus.cc:348] T b037293d5b60471082da209381a3e9bf P 4900b3066faf472f9b7e666fe9f91d25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } }
I20260811 18:40:25.744541 1800 raft_consensus.cc:374] T b037293d5b60471082da209381a3e9bf P 4900b3066faf472f9b7e666fe9f91d25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:25.744745 1800 raft_consensus.cc:729] T b037293d5b60471082da209381a3e9bf P 4900b3066faf472f9b7e666fe9f91d25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4900b3066faf472f9b7e666fe9f91d25, State: Initialized, Role: FOLLOWER
I20260811 18:40:25.746491 1753 heartbeater.cc:503] Master 127.30.185.190:44247 was elected leader, sending a full tablet report...
I20260811 18:40:25.747133 1798 ts_tablet_manager.cc:1428] T b037293d5b60471082da209381a3e9bf P 4fb98c59b7d94d94af075f92b4bdd94a: Time spent starting tablet: real 0.024s user 0.011s sys 0.012s
I20260811 18:40:25.747900 1800 consensus_queue.cc:260] T b037293d5b60471082da209381a3e9bf P 4900b3066faf472f9b7e666fe9f91d25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } }
I20260811 18:40:25.749609 1493 heartbeater.cc:503] Master 127.30.185.190:44247 was elected leader, sending a full tablet report...
I20260811 18:40:25.750366 1799 ts_tablet_manager.cc:1428] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762: Time spent starting tablet: real 0.027s user 0.020s sys 0.008s
I20260811 18:40:25.757233 1800 ts_tablet_manager.cc:1428] T b037293d5b60471082da209381a3e9bf P 4900b3066faf472f9b7e666fe9f91d25: Time spent starting tablet: real 0.028s user 0.020s sys 0.005s
W20260811 18:40:25.809837 1754 tablet.cc:2369] T b037293d5b60471082da209381a3e9bf P 4fb98c59b7d94d94af075f92b4bdd94a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:40:25.854204 1810 raft_consensus.cc:482] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:40:25.854688 1810 raft_consensus.cc:504] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } }
I20260811 18:40:25.856770 1810 leader_election.cc:288] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4fb98c59b7d94d94af075f92b4bdd94a (127.30.185.131:40971), 4900b3066faf472f9b7e666fe9f91d25 (127.30.185.129:34513)
I20260811 18:40:25.865502 1784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:25.857073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:46961 (local address 127.30.185.131:40971)
0811 18:40:25.857383 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:25.857391 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:25.858349 (+ 958us) server_negotiation.cc:407] Connection header received
0811 18:40:25.858621 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:25.858629 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:25.858753 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:25.858896 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:25.859664 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.860792 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.863003 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.864296 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.864430 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:25.864484 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:25.865041 (+ 557us) server_negotiation.cc:299] Negotiation successful
0811 18:40:25.865227 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 18:40:25.865629 1792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:25.857401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:45533 (local address 127.30.185.129:34513)
0811 18:40:25.857673 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:25.857681 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:25.859098 (+ 1417us) server_negotiation.cc:407] Connection header received
0811 18:40:25.859329 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:25.859336 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:25.859415 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:25.859523 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:25.860338 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.861358 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.863600 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.864570 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.864735 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:25.864814 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:25.865213 (+ 399us) server_negotiation.cc:299] Negotiation successful
0811 18:40:25.865366 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 18:40:25.866158 1815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:25.857026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:40971 (local address 127.30.185.130:46961)
0811 18:40:25.858179 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:25.858206 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:25.858412 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:25.858922 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:25.858925 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:25.858953 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:25.859486 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:25.859503 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.860935 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.860941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:25.862829 (+ 1888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:25.862841 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.864591 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.864599 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:25.864873 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:25.864899 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 18:40:25.864973 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 18:40:25.865068 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":983,"mutex_wait_us":24,"spinlock_wait_cycles":18816,"thread_start_us":617,"threads_started":1}
I20260811 18:40:25.866240 1816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:25.857863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:34513 (local address 127.30.185.130:45533)
0811 18:40:25.858951 (+ 1088us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:25.858976 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:25.859149 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:25.859548 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:25.859551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:25.859605 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:25.860155 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:25.860171 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.861505 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.861508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:25.863411 (+ 1903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:25.863426 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.864819 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.864824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:25.865008 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:25.865042 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 18:40:25.865141 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 18:40:25.865296 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":954,"mutex_wait_us":42,"thread_start_us":713,"threads_started":1}
I20260811 18:40:25.867515 1190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b037293d5b60471082da209381a3e9bf" candidate_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4900b3066faf472f9b7e666fe9f91d25" is_pre_election: true
I20260811 18:40:25.867585 1671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b037293d5b60471082da209381a3e9bf" candidate_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" is_pre_election: true
I20260811 18:40:25.868120 1190 raft_consensus.cc:2454] T b037293d5b60471082da209381a3e9bf P 4900b3066faf472f9b7e666fe9f91d25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 721f910c1ab04a5da8a0cfedfd0bf762 in term 0.
I20260811 18:40:25.868208 1671 raft_consensus.cc:2454] T b037293d5b60471082da209381a3e9bf P 4fb98c59b7d94d94af075f92b4bdd94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 721f910c1ab04a5da8a0cfedfd0bf762 in term 0.
I20260811 18:40:25.868911 1304 leader_election.cc:302] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4900b3066faf472f9b7e666fe9f91d25, 721f910c1ab04a5da8a0cfedfd0bf762; no voters:
I20260811 18:40:25.869376 1810 raft_consensus.cc:2781] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:40:25.869537 1810 raft_consensus.cc:482] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:40:25.869654 1810 raft_consensus.cc:3037] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:25.872102 1810 raft_consensus.cc:504] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } }
I20260811 18:40:25.872798 1810 leader_election.cc:288] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762 [CANDIDATE]: Term 1 election: Requested vote from peers 4fb98c59b7d94d94af075f92b4bdd94a (127.30.185.131:40971), 4900b3066faf472f9b7e666fe9f91d25 (127.30.185.129:34513)
I20260811 18:40:25.873276 1671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b037293d5b60471082da209381a3e9bf" candidate_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fb98c59b7d94d94af075f92b4bdd94a"
I20260811 18:40:25.873389 1190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b037293d5b60471082da209381a3e9bf" candidate_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4900b3066faf472f9b7e666fe9f91d25"
I20260811 18:40:25.873544 1671 raft_consensus.cc:3037] T b037293d5b60471082da209381a3e9bf P 4fb98c59b7d94d94af075f92b4bdd94a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:25.873592 1190 raft_consensus.cc:3037] T b037293d5b60471082da209381a3e9bf P 4900b3066faf472f9b7e666fe9f91d25 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:25.877282 1671 raft_consensus.cc:2454] T b037293d5b60471082da209381a3e9bf P 4fb98c59b7d94d94af075f92b4bdd94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 721f910c1ab04a5da8a0cfedfd0bf762 in term 1.
I20260811 18:40:25.877282 1190 raft_consensus.cc:2454] T b037293d5b60471082da209381a3e9bf P 4900b3066faf472f9b7e666fe9f91d25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 721f910c1ab04a5da8a0cfedfd0bf762 in term 1.
I20260811 18:40:25.877790 1306 leader_election.cc:302] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fb98c59b7d94d94af075f92b4bdd94a, 721f910c1ab04a5da8a0cfedfd0bf762; no voters:
I20260811 18:40:25.878106 1810 raft_consensus.cc:2781] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:40:25.878643 1810 raft_consensus.cc:686] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762 [term 1 LEADER]: Becoming Leader. State: Replica: 721f910c1ab04a5da8a0cfedfd0bf762, State: Running, Role: LEADER
I20260811 18:40:25.879199 1810 consensus_queue.cc:237] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } }
I20260811 18:40:25.884521 927 catalog_manager.cc:5495] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762 reported cstate change: term changed from 0 to 1, leader changed from <none> to 721f910c1ab04a5da8a0cfedfd0bf762 (127.30.185.130). New cstate: current_term: 1 leader_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } health_report { overall_health: UNKNOWN } } }
I20260811 18:40:25.924417 1796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:25.918232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45112 (local address 127.30.185.130:33361)
0811 18:40:25.918426 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:25.918431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:25.918738 (+ 307us) server_negotiation.cc:407] Connection header received
0811 18:40:25.919108 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:25.919113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:25.919215 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:25.919364 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:40:25.919782 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.920550 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.922193 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:25.922614 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:25.922716 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:25.923221 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:40:25.923838 (+ 617us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:40:25.923953 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:25.924122 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 18:40:25.924193 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 18:40:25.934336 1190 raft_consensus.cc:1261] T b037293d5b60471082da209381a3e9bf P 4900b3066faf472f9b7e666fe9f91d25 [term 1 FOLLOWER]: Refusing update from remote peer 721f910c1ab04a5da8a0cfedfd0bf762: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:40:25.934353 1671 raft_consensus.cc:1261] T b037293d5b60471082da209381a3e9bf P 4fb98c59b7d94d94af075f92b4bdd94a [term 1 FOLLOWER]: Refusing update from remote peer 721f910c1ab04a5da8a0cfedfd0bf762: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260811 18:40:25.935545 1273 tablet.cc:2369] T b037293d5b60471082da209381a3e9bf P 4900b3066faf472f9b7e666fe9f91d25: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:40:25.935634 1817 consensus_queue.cc:1035] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:40:25.936184 1810 consensus_queue.cc:1035] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:40:25.974056 926 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:40:25.974455 926 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:40:25.979562 926 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988: Sending DeleteTablet for 3 replicas of tablet b037293d5b60471082da209381a3e9bf
I20260811 18:40:25.981091 1625 tablet_service.cc:1514] Processing DeleteTablet for tablet b037293d5b60471082da209381a3e9bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:25 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:33258
I20260811 18:40:25.981417 1157 tablet_service.cc:1514] Processing DeleteTablet for tablet b037293d5b60471082da209381a3e9bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:25 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:53532
I20260811 18:40:25.982120 1385 tablet_service.cc:1514] Processing DeleteTablet for tablet b037293d5b60471082da209381a3e9bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:25 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:45098
I20260811 18:40:25.983554 926 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39244:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:40:25.984510 926 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:40:25.987900 1852 tablet_replica.cc:330] T b037293d5b60471082da209381a3e9bf P 4900b3066faf472f9b7e666fe9f91d25: stopping tablet replica
I20260811 18:40:25.988744 1852 raft_consensus.cc:2229] T b037293d5b60471082da209381a3e9bf P 4900b3066faf472f9b7e666fe9f91d25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:40:25.989346 1852 raft_consensus.cc:2258] T b037293d5b60471082da209381a3e9bf P 4900b3066faf472f9b7e666fe9f91d25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:40:25.989126 1853 tablet_replica.cc:330] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762: stopping tablet replica
I20260811 18:40:25.992530 1853 raft_consensus.cc:2229] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:40:25.993566 1853 raft_consensus.cc:2258] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:40:25.994999 1852 ts_tablet_manager.cc:1905] T b037293d5b60471082da209381a3e9bf P 4900b3066faf472f9b7e666fe9f91d25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:40:25.995836 1621 tablet_service.cc:1467] Processing CreateTablet for tablet 1a180b8209df410ca3fc1c57c7725ed1 (DEFAULT_TABLE table=test-table [id=582589b95bb143df8501761d4eb3bac7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:25.996547 1621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a180b8209df410ca3fc1c57c7725ed1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:25.995328 1157 tablet_service.cc:1467] Processing CreateTablet for tablet 1a180b8209df410ca3fc1c57c7725ed1 (DEFAULT_TABLE table=test-table [id=582589b95bb143df8501761d4eb3bac7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:25.998993 1157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a180b8209df410ca3fc1c57c7725ed1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:26.003013 1845 tablet_replica.cc:330] T b037293d5b60471082da209381a3e9bf P 4fb98c59b7d94d94af075f92b4bdd94a: stopping tablet replica
I20260811 18:40:26.003026 1385 tablet_service.cc:1467] Processing CreateTablet for tablet 1a180b8209df410ca3fc1c57c7725ed1 (DEFAULT_TABLE table=test-table [id=582589b95bb143df8501761d4eb3bac7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:26.003509 1845 raft_consensus.cc:2229] T b037293d5b60471082da209381a3e9bf P 4fb98c59b7d94d94af075f92b4bdd94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:40:26.003615 1385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a180b8209df410ca3fc1c57c7725ed1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:26.004199 1845 raft_consensus.cc:2258] T b037293d5b60471082da209381a3e9bf P 4fb98c59b7d94d94af075f92b4bdd94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:40:26.008383 1853 ts_tablet_manager.cc:1905] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:40:26.011065 1800 tablet_bootstrap.cc:492] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25: Bootstrap starting.
I20260811 18:40:26.011955 1798 tablet_bootstrap.cc:492] T 1a180b8209df410ca3fc1c57c7725ed1 P 4fb98c59b7d94d94af075f92b4bdd94a: Bootstrap starting.
I20260811 18:40:26.013018 1800 tablet_bootstrap.cc:654] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:26.013947 1798 tablet_bootstrap.cc:654] T 1a180b8209df410ca3fc1c57c7725ed1 P 4fb98c59b7d94d94af075f92b4bdd94a: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:26.018420 1845 ts_tablet_manager.cc:1905] T b037293d5b60471082da209381a3e9bf P 4fb98c59b7d94d94af075f92b4bdd94a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:40:26.023059 1800 tablet_bootstrap.cc:492] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25: No bootstrap required, opened a new log
I20260811 18:40:26.023272 1800 ts_tablet_manager.cc:1397] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.004s
I20260811 18:40:26.024435 1800 raft_consensus.cc:348] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } }
I20260811 18:40:26.024686 1800 raft_consensus.cc:374] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:26.024844 1800 raft_consensus.cc:729] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4900b3066faf472f9b7e666fe9f91d25, State: Initialized, Role: FOLLOWER
I20260811 18:40:26.025269 1800 consensus_queue.cc:260] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } }
I20260811 18:40:26.027288 1845 ts_tablet_manager.cc:1918] T b037293d5b60471082da209381a3e9bf P 4fb98c59b7d94d94af075f92b4bdd94a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:40:26.027542 1845 log.cc:1196] T b037293d5b60471082da209381a3e9bf P 4fb98c59b7d94d94af075f92b4bdd94a: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/b037293d5b60471082da209381a3e9bf
I20260811 18:40:26.028236 1845 ts_tablet_manager.cc:1939] T b037293d5b60471082da209381a3e9bf P 4fb98c59b7d94d94af075f92b4bdd94a: Deleting consensus metadata
I20260811 18:40:26.028774 1800 ts_tablet_manager.cc:1428] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25: Time spent starting tablet: real 0.005s user 0.001s sys 0.002s
I20260811 18:40:26.030722 897 catalog_manager.cc:4841] TS 4fb98c59b7d94d94af075f92b4bdd94a (127.30.185.131:40971): tablet b037293d5b60471082da209381a3e9bf (table test-table [id=787e9cbe53834b05816bc4dbf258d46a]) successfully deleted
I20260811 18:40:26.032732 1799 tablet_bootstrap.cc:492] T 1a180b8209df410ca3fc1c57c7725ed1 P 721f910c1ab04a5da8a0cfedfd0bf762: Bootstrap starting.
I20260811 18:40:26.033463 1852 ts_tablet_manager.cc:1918] T b037293d5b60471082da209381a3e9bf P 4900b3066faf472f9b7e666fe9f91d25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:40:26.033704 1852 log.cc:1196] T b037293d5b60471082da209381a3e9bf P 4900b3066faf472f9b7e666fe9f91d25: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-0/wal/wals/b037293d5b60471082da209381a3e9bf
I20260811 18:40:26.034364 1852 ts_tablet_manager.cc:1939] T b037293d5b60471082da209381a3e9bf P 4900b3066faf472f9b7e666fe9f91d25: Deleting consensus metadata
I20260811 18:40:26.036234 1798 tablet_bootstrap.cc:492] T 1a180b8209df410ca3fc1c57c7725ed1 P 4fb98c59b7d94d94af075f92b4bdd94a: No bootstrap required, opened a new log
I20260811 18:40:26.036571 1798 ts_tablet_manager.cc:1397] T 1a180b8209df410ca3fc1c57c7725ed1 P 4fb98c59b7d94d94af075f92b4bdd94a: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.011s
I20260811 18:40:26.037197 895 catalog_manager.cc:4841] TS 4900b3066faf472f9b7e666fe9f91d25 (127.30.185.129:34513): tablet b037293d5b60471082da209381a3e9bf (table test-table [id=787e9cbe53834b05816bc4dbf258d46a]) successfully deleted
I20260811 18:40:26.037922 1798 raft_consensus.cc:348] T 1a180b8209df410ca3fc1c57c7725ed1 P 4fb98c59b7d94d94af075f92b4bdd94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } }
I20260811 18:40:26.038260 1798 raft_consensus.cc:374] T 1a180b8209df410ca3fc1c57c7725ed1 P 4fb98c59b7d94d94af075f92b4bdd94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:26.038762 1798 raft_consensus.cc:729] T 1a180b8209df410ca3fc1c57c7725ed1 P 4fb98c59b7d94d94af075f92b4bdd94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fb98c59b7d94d94af075f92b4bdd94a, State: Initialized, Role: FOLLOWER
I20260811 18:40:26.039187 1798 consensus_queue.cc:260] T 1a180b8209df410ca3fc1c57c7725ed1 P 4fb98c59b7d94d94af075f92b4bdd94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } }
I20260811 18:40:26.041181 1798 ts_tablet_manager.cc:1428] T 1a180b8209df410ca3fc1c57c7725ed1 P 4fb98c59b7d94d94af075f92b4bdd94a: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:40:26.043674 1799 tablet_bootstrap.cc:654] T 1a180b8209df410ca3fc1c57c7725ed1 P 721f910c1ab04a5da8a0cfedfd0bf762: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:26.055372 1799 tablet_bootstrap.cc:492] T 1a180b8209df410ca3fc1c57c7725ed1 P 721f910c1ab04a5da8a0cfedfd0bf762: No bootstrap required, opened a new log
I20260811 18:40:26.055567 1799 ts_tablet_manager.cc:1397] T 1a180b8209df410ca3fc1c57c7725ed1 P 721f910c1ab04a5da8a0cfedfd0bf762: Time spent bootstrapping tablet: real 0.023s user 0.016s sys 0.003s
I20260811 18:40:26.056412 1853 ts_tablet_manager.cc:1918] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:40:26.056897 1853 log.cc:1196] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/b037293d5b60471082da209381a3e9bf
I20260811 18:40:26.056967 1799 raft_consensus.cc:348] T 1a180b8209df410ca3fc1c57c7725ed1 P 721f910c1ab04a5da8a0cfedfd0bf762 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } }
I20260811 18:40:26.057529 1799 raft_consensus.cc:374] T 1a180b8209df410ca3fc1c57c7725ed1 P 721f910c1ab04a5da8a0cfedfd0bf762 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:26.057698 1799 raft_consensus.cc:729] T 1a180b8209df410ca3fc1c57c7725ed1 P 721f910c1ab04a5da8a0cfedfd0bf762 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 721f910c1ab04a5da8a0cfedfd0bf762, State: Initialized, Role: FOLLOWER
I20260811 18:40:26.057917 1853 ts_tablet_manager.cc:1939] T b037293d5b60471082da209381a3e9bf P 721f910c1ab04a5da8a0cfedfd0bf762: Deleting consensus metadata
I20260811 18:40:26.058002 1799 consensus_queue.cc:260] T 1a180b8209df410ca3fc1c57c7725ed1 P 721f910c1ab04a5da8a0cfedfd0bf762 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } }
I20260811 18:40:26.058825 1799 ts_tablet_manager.cc:1428] T 1a180b8209df410ca3fc1c57c7725ed1 P 721f910c1ab04a5da8a0cfedfd0bf762: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:40:26.061153 897 catalog_manager.cc:4841] TS 721f910c1ab04a5da8a0cfedfd0bf762 (127.30.185.130:33361): tablet b037293d5b60471082da209381a3e9bf (table test-table [id=787e9cbe53834b05816bc4dbf258d46a]) successfully deleted
I20260811 18:40:26.126986 1812 raft_consensus.cc:482] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:40:26.127243 1812 raft_consensus.cc:504] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } }
I20260811 18:40:26.129011 1812 leader_election.cc:288] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 721f910c1ab04a5da8a0cfedfd0bf762 (127.30.185.130:33361), 4fb98c59b7d94d94af075f92b4bdd94a (127.30.185.131:40971)
I20260811 18:40:26.139245 1868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:26.130706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:40971 (local address 127.30.185.129:48279)
0811 18:40:26.131703 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:26.131721 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:26.131903 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:26.132314 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:26.132318 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:26.132345 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:26.132968 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:26.132986 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.134566 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.134572 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:26.136796 (+ 2224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:26.136807 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.138061 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.138068 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:26.138229 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:26.138254 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:40:26.138362 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 18:40:26.138496 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"thread_start_us":639,"threads_started":1}
I20260811 18:40:26.139859 1796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:26.129486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:37161 (local address 127.30.185.130:33361)
0811 18:40:26.129664 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:26.129669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:26.130669 (+ 1000us) server_negotiation.cc:407] Connection header received
0811 18:40:26.130995 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:26.131001 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:26.131086 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:26.131168 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:26.132295 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.133709 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.136751 (+ 3042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.138264 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.138422 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:26.138506 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:26.139397 (+ 891us) server_negotiation.cc:299] Negotiation successful
0811 18:40:26.139555 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 18:40:26.138899 1784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:26.130902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:48279 (local address 127.30.185.131:40971)
0811 18:40:26.131082 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:26.131088 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:26.131871 (+ 783us) server_negotiation.cc:407] Connection header received
0811 18:40:26.132064 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:26.132070 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:26.132150 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:26.132250 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:26.133205 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.134396 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.136942 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.137850 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.138031 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:26.138111 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:26.138399 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 18:40:26.138527 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 18:40:26.140584 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:26.129713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:33361 (local address 127.30.185.129:37161)
0811 18:40:26.130526 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:26.130543 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:26.130800 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:26.131481 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:26.131486 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:26.131519 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:26.132103 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:26.132114 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.133857 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.133865 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:26.136563 (+ 2698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:26.136579 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.138904 (+ 2325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.138912 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:26.139177 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:26.139218 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 18:40:26.140052 (+ 834us) client_negotiation.cc:240] Negotiation successful
0811 18:40:26.140202 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":460,"threads_started":1}
I20260811 18:40:26.141415 1671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a180b8209df410ca3fc1c57c7725ed1" candidate_uuid: "4900b3066faf472f9b7e666fe9f91d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" is_pre_election: true
I20260811 18:40:26.141731 1671 raft_consensus.cc:2454] T 1a180b8209df410ca3fc1c57c7725ed1 P 4fb98c59b7d94d94af075f92b4bdd94a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4900b3066faf472f9b7e666fe9f91d25 in term 0.
I20260811 18:40:26.141922 1416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a180b8209df410ca3fc1c57c7725ed1" candidate_uuid: "4900b3066faf472f9b7e666fe9f91d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" is_pre_election: true
I20260811 18:40:26.142426 1416 raft_consensus.cc:2454] T 1a180b8209df410ca3fc1c57c7725ed1 P 721f910c1ab04a5da8a0cfedfd0bf762 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4900b3066faf472f9b7e666fe9f91d25 in term 0.
I20260811 18:40:26.143087 1068 leader_election.cc:302] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4900b3066faf472f9b7e666fe9f91d25, 721f910c1ab04a5da8a0cfedfd0bf762; no voters:
I20260811 18:40:26.143730 1812 raft_consensus.cc:2781] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:40:26.143913 1812 raft_consensus.cc:482] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:40:26.144042 1812 raft_consensus.cc:3037] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:26.146682 1812 raft_consensus.cc:504] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } }
I20260811 18:40:26.147398 1812 leader_election.cc:288] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25 [CANDIDATE]: Term 1 election: Requested vote from peers 721f910c1ab04a5da8a0cfedfd0bf762 (127.30.185.130:33361), 4fb98c59b7d94d94af075f92b4bdd94a (127.30.185.131:40971)
I20260811 18:40:26.147837 1416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a180b8209df410ca3fc1c57c7725ed1" candidate_uuid: "4900b3066faf472f9b7e666fe9f91d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "721f910c1ab04a5da8a0cfedfd0bf762"
I20260811 18:40:26.147897 1671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a180b8209df410ca3fc1c57c7725ed1" candidate_uuid: "4900b3066faf472f9b7e666fe9f91d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fb98c59b7d94d94af075f92b4bdd94a"
I20260811 18:40:26.148051 1416 raft_consensus.cc:3037] T 1a180b8209df410ca3fc1c57c7725ed1 P 721f910c1ab04a5da8a0cfedfd0bf762 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:26.148149 1671 raft_consensus.cc:3037] T 1a180b8209df410ca3fc1c57c7725ed1 P 4fb98c59b7d94d94af075f92b4bdd94a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:26.150663 1416 raft_consensus.cc:2454] T 1a180b8209df410ca3fc1c57c7725ed1 P 721f910c1ab04a5da8a0cfedfd0bf762 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4900b3066faf472f9b7e666fe9f91d25 in term 1.
I20260811 18:40:26.151114 1068 leader_election.cc:302] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4900b3066faf472f9b7e666fe9f91d25, 721f910c1ab04a5da8a0cfedfd0bf762; no voters:
I20260811 18:40:26.151430 1812 raft_consensus.cc:2781] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:40:26.151880 1671 raft_consensus.cc:2454] T 1a180b8209df410ca3fc1c57c7725ed1 P 4fb98c59b7d94d94af075f92b4bdd94a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4900b3066faf472f9b7e666fe9f91d25 in term 1.
I20260811 18:40:26.151983 1812 raft_consensus.cc:686] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25 [term 1 LEADER]: Becoming Leader. State: Replica: 4900b3066faf472f9b7e666fe9f91d25, State: Running, Role: LEADER
I20260811 18:40:26.152516 1812 consensus_queue.cc:237] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } }
I20260811 18:40:26.156862 926 catalog_manager.cc:5495] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4900b3066faf472f9b7e666fe9f91d25 (127.30.185.129). New cstate: current_term: 1 leader_uuid: "4900b3066faf472f9b7e666fe9f91d25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } health_report { overall_health: UNKNOWN } } }
Aug 11 18:40:26 dist-test-slave-z8x9 krb5kdc[855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473623, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:26.220932 1880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:26.199598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:44247 (local address 127.0.0.1:39246)
0811 18:40:26.200581 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:26.200600 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:26.200728 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:26.201115 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:26.201119 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:26.202063 (+ 944us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:26.202325 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:26.202336 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.203900 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.203904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:26.205391 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:26.205405 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.206517 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.206521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:26.206695 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:26.210934 (+ 4239us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:26.210954 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:26.214023 (+ 3069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:26.217696 (+ 3673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:26.217714 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:26.217722 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:26.218460 (+ 738us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:26.218882 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:26.218888 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:26.218893 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:26.219077 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:26.219662 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:26.219670 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:26.219904 (+ 234us) client_negotiation.cc:769] Sending connection context
0811 18:40:26.220141 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 18:40:26.220499 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":836,"thread_start_us":703,"threads_started":1}
I20260811 18:40:26.221282 1878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:26.199641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39246 (local address 127.30.185.190:44247)
0811 18:40:26.200311 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:26.200317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:26.200694 (+ 377us) server_negotiation.cc:407] Connection header received
0811 18:40:26.200851 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:26.200857 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:26.200942 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:26.201067 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:26.202491 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.203762 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.205542 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.206352 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.206492 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:26.214294 (+ 7802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:26.214300 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:26.214304 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:26.214336 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:26.217552 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:26.218597 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:26.218602 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:26.218604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:26.218761 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:26.219189 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:26.219193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:26.219195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:26.219512 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:26.219626 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:26.220269 (+ 643us) server_negotiation.cc:299] Negotiation successful
0811 18:40:26.220631 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":155,"server-negotiator.queue_time_us":437,"thread_start_us":329,"threads_started":1}
I20260811 18:40:26.227242 926 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:39246:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 18:40:26.228612 926 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:40:26.237455 1157 tablet_service.cc:1467] Processing CreateTablet for tablet a08252699ba541e9b312342cae4cf8d4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d255bf3127d54b66b3fbd1daa03f84f4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:40:26.237519 1385 tablet_service.cc:1467] Processing CreateTablet for tablet a08252699ba541e9b312342cae4cf8d4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d255bf3127d54b66b3fbd1daa03f84f4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:40:26.238056 1157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a08252699ba541e9b312342cae4cf8d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:26.238227 1385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a08252699ba541e9b312342cae4cf8d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:26.243479 1621 tablet_service.cc:1467] Processing CreateTablet for tablet a08252699ba541e9b312342cae4cf8d4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d255bf3127d54b66b3fbd1daa03f84f4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:40:26.244014 1621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a08252699ba541e9b312342cae4cf8d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:26.246793 1799 tablet_bootstrap.cc:492] T a08252699ba541e9b312342cae4cf8d4 P 721f910c1ab04a5da8a0cfedfd0bf762: Bootstrap starting.
W20260811 18:40:26.248827 1494 tablet.cc:2369] T 1a180b8209df410ca3fc1c57c7725ed1 P 721f910c1ab04a5da8a0cfedfd0bf762: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:40:26.249679 1799 tablet_bootstrap.cc:654] T a08252699ba541e9b312342cae4cf8d4 P 721f910c1ab04a5da8a0cfedfd0bf762: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:26.250344 1798 tablet_bootstrap.cc:492] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a: Bootstrap starting.
I20260811 18:40:26.252085 1800 tablet_bootstrap.cc:492] T a08252699ba541e9b312342cae4cf8d4 P 4900b3066faf472f9b7e666fe9f91d25: Bootstrap starting.
I20260811 18:40:26.252260 1798 tablet_bootstrap.cc:654] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:26.254580 1800 tablet_bootstrap.cc:654] T a08252699ba541e9b312342cae4cf8d4 P 4900b3066faf472f9b7e666fe9f91d25: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:26.255467 1798 tablet_bootstrap.cc:492] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a: No bootstrap required, opened a new log
I20260811 18:40:26.255868 1798 ts_tablet_manager.cc:1397] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 18:40:26.257205 1798 raft_consensus.cc:348] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } }
I20260811 18:40:26.257622 1798 raft_consensus.cc:374] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:26.257725 1798 raft_consensus.cc:729] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fb98c59b7d94d94af075f92b4bdd94a, State: Initialized, Role: FOLLOWER
I20260811 18:40:26.258029 1798 consensus_queue.cc:260] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } }
I20260811 18:40:26.258553 1809 tablet_replica.cc:439] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } } }
I20260811 18:40:26.258968 1809 tablet_replica.cc:442] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:40:26.262578 1798 ts_tablet_manager.cc:1428] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260811 18:40:26.265515 1800 tablet_bootstrap.cc:492] T a08252699ba541e9b312342cae4cf8d4 P 4900b3066faf472f9b7e666fe9f91d25: No bootstrap required, opened a new log
I20260811 18:40:26.265740 1800 ts_tablet_manager.cc:1397] T a08252699ba541e9b312342cae4cf8d4 P 4900b3066faf472f9b7e666fe9f91d25: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260811 18:40:26.265827 1799 tablet_bootstrap.cc:492] T a08252699ba541e9b312342cae4cf8d4 P 721f910c1ab04a5da8a0cfedfd0bf762: No bootstrap required, opened a new log
I20260811 18:40:26.266077 1799 ts_tablet_manager.cc:1397] T a08252699ba541e9b312342cae4cf8d4 P 721f910c1ab04a5da8a0cfedfd0bf762: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.005s
I20260811 18:40:26.266541 1800 raft_consensus.cc:348] T a08252699ba541e9b312342cae4cf8d4 P 4900b3066faf472f9b7e666fe9f91d25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } }
I20260811 18:40:26.266857 1800 raft_consensus.cc:374] T a08252699ba541e9b312342cae4cf8d4 P 4900b3066faf472f9b7e666fe9f91d25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:26.266984 1800 raft_consensus.cc:729] T a08252699ba541e9b312342cae4cf8d4 P 4900b3066faf472f9b7e666fe9f91d25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4900b3066faf472f9b7e666fe9f91d25, State: Initialized, Role: FOLLOWER
I20260811 18:40:26.266887 1799 raft_consensus.cc:348] T a08252699ba541e9b312342cae4cf8d4 P 721f910c1ab04a5da8a0cfedfd0bf762 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } }
I20260811 18:40:26.267199 1799 raft_consensus.cc:374] T a08252699ba541e9b312342cae4cf8d4 P 721f910c1ab04a5da8a0cfedfd0bf762 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:26.267299 1799 raft_consensus.cc:729] T a08252699ba541e9b312342cae4cf8d4 P 721f910c1ab04a5da8a0cfedfd0bf762 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 721f910c1ab04a5da8a0cfedfd0bf762, State: Initialized, Role: FOLLOWER
I20260811 18:40:26.267387 1800 consensus_queue.cc:260] T a08252699ba541e9b312342cae4cf8d4 P 4900b3066faf472f9b7e666fe9f91d25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } }
I20260811 18:40:26.267575 1799 consensus_queue.cc:260] T a08252699ba541e9b312342cae4cf8d4 P 721f910c1ab04a5da8a0cfedfd0bf762 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } }
I20260811 18:40:26.268033 1800 ts_tablet_manager.cc:1428] T a08252699ba541e9b312342cae4cf8d4 P 4900b3066faf472f9b7e666fe9f91d25: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:40:26.268239 1799 ts_tablet_manager.cc:1428] T a08252699ba541e9b312342cae4cf8d4 P 721f910c1ab04a5da8a0cfedfd0bf762: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 18:40:26.268537 1812 tablet_replica.cc:439] T a08252699ba541e9b312342cae4cf8d4 P 4900b3066faf472f9b7e666fe9f91d25: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } } }
I20260811 18:40:26.268898 1812 tablet_replica.cc:442] T a08252699ba541e9b312342cae4cf8d4 P 4900b3066faf472f9b7e666fe9f91d25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:40:26.269701 1810 tablet_replica.cc:439] T a08252699ba541e9b312342cae4cf8d4 P 721f910c1ab04a5da8a0cfedfd0bf762: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } } }
I20260811 18:40:26.269989 1810 tablet_replica.cc:442] T a08252699ba541e9b312342cae4cf8d4 P 721f910c1ab04a5da8a0cfedfd0bf762: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:40:26.317710 1809 raft_consensus.cc:482] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:40:26.318194 1809 raft_consensus.cc:504] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } }
I20260811 18:40:26.322489 1809 leader_election.cc:288] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4900b3066faf472f9b7e666fe9f91d25 (127.30.185.129:34513), 721f910c1ab04a5da8a0cfedfd0bf762 (127.30.185.130:33361)
I20260811 18:40:26.340950 1888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:26.322999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:33361 (local address 127.30.185.131:49415)
0811 18:40:26.327741 (+ 4742us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:26.327768 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:26.328004 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:26.328791 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:26.328799 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:26.328880 (+ 81us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:26.330364 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:26.330396 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.333333 (+ 2937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.333349 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:26.336718 (+ 3369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:26.336738 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.339055 (+ 2317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.339072 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:26.339455 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:26.339592 (+ 137us) client_negotiation.cc:769] Sending connection context
0811 18:40:26.339807 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 18:40:26.340069 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4571,"mutex_wait_us":48,"thread_start_us":4287,"threads_started":1}
I20260811 18:40:26.341279 1885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:26.322589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:34513 (local address 127.30.185.131:47039)
0811 18:40:26.327210 (+ 4621us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:26.327284 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:26.327731 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:26.328659 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:26.328677 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:26.328752 (+ 75us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:26.329614 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:26.329635 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.331471 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.331490 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:26.335293 (+ 3803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:26.335452 (+ 159us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.339481 (+ 4029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.339502 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:26.339973 (+ 471us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:26.340029 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 18:40:26.340183 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 18:40:26.340407 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4216,"mutex_wait_us":2,"thread_start_us":3845,"threads_started":1}
I20260811 18:40:26.342064 1792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:26.323368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:47039 (local address 127.30.185.129:34513)
0811 18:40:26.323794 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:26.323806 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:26.327468 (+ 3662us) server_negotiation.cc:407] Connection header received
0811 18:40:26.328119 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:26.328136 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:26.328289 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:26.328436 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:26.330105 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.331268 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.335747 (+ 4479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.338866 (+ 3119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.339169 (+ 303us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:26.339338 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:26.340313 (+ 975us) server_negotiation.cc:299] Negotiation successful
0811 18:40:26.340646 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260811 18:40:26.342389 1796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:26.323548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:49415 (local address 127.30.185.130:33361)
0811 18:40:26.324010 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:26.324032 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:26.327904 (+ 3872us) server_negotiation.cc:407] Connection header received
0811 18:40:26.328249 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:26.328261 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:26.328348 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:26.328560 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:26.330738 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.333063 (+ 2325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.337053 (+ 3990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.338715 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.339075 (+ 360us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:26.339197 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:26.341612 (+ 2415us) server_negotiation.cc:299] Negotiation successful
0811 18:40:26.341907 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260811 18:40:26.343878 1416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08252699ba541e9b312342cae4cf8d4" candidate_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" is_pre_election: true
I20260811 18:40:26.343914 1190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08252699ba541e9b312342cae4cf8d4" candidate_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4900b3066faf472f9b7e666fe9f91d25" is_pre_election: true
I20260811 18:40:26.344340 1416 raft_consensus.cc:2454] T a08252699ba541e9b312342cae4cf8d4 P 721f910c1ab04a5da8a0cfedfd0bf762 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fb98c59b7d94d94af075f92b4bdd94a in term 0.
I20260811 18:40:26.344374 1190 raft_consensus.cc:2454] T a08252699ba541e9b312342cae4cf8d4 P 4900b3066faf472f9b7e666fe9f91d25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fb98c59b7d94d94af075f92b4bdd94a in term 0.
I20260811 18:40:26.345562 1535 leader_election.cc:302] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4900b3066faf472f9b7e666fe9f91d25, 4fb98c59b7d94d94af075f92b4bdd94a, 721f910c1ab04a5da8a0cfedfd0bf762; no voters:
I20260811 18:40:26.346484 1809 raft_consensus.cc:2781] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:40:26.346820 1809 raft_consensus.cc:482] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:40:26.347076 1809 raft_consensus.cc:3037] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:26.351817 1809 raft_consensus.cc:504] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } }
I20260811 18:40:26.353992 1809 leader_election.cc:288] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a [CANDIDATE]: Term 1 election: Requested vote from peers 4900b3066faf472f9b7e666fe9f91d25 (127.30.185.129:34513), 721f910c1ab04a5da8a0cfedfd0bf762 (127.30.185.130:33361)
I20260811 18:40:26.354135 1416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08252699ba541e9b312342cae4cf8d4" candidate_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "721f910c1ab04a5da8a0cfedfd0bf762"
I20260811 18:40:26.354650 1416 raft_consensus.cc:3037] T a08252699ba541e9b312342cae4cf8d4 P 721f910c1ab04a5da8a0cfedfd0bf762 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:26.354481 1190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08252699ba541e9b312342cae4cf8d4" candidate_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4900b3066faf472f9b7e666fe9f91d25"
I20260811 18:40:26.355110 1190 raft_consensus.cc:3037] T a08252699ba541e9b312342cae4cf8d4 P 4900b3066faf472f9b7e666fe9f91d25 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:26.359109 1416 raft_consensus.cc:2454] T a08252699ba541e9b312342cae4cf8d4 P 721f910c1ab04a5da8a0cfedfd0bf762 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fb98c59b7d94d94af075f92b4bdd94a in term 1.
I20260811 18:40:26.359915 1537 leader_election.cc:302] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fb98c59b7d94d94af075f92b4bdd94a, 721f910c1ab04a5da8a0cfedfd0bf762; no voters:
I20260811 18:40:26.360530 1809 raft_consensus.cc:2781] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:40:26.361433 1809 raft_consensus.cc:686] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a [term 1 LEADER]: Becoming Leader. State: Replica: 4fb98c59b7d94d94af075f92b4bdd94a, State: Running, Role: LEADER
I20260811 18:40:26.361785 1190 raft_consensus.cc:2454] T a08252699ba541e9b312342cae4cf8d4 P 4900b3066faf472f9b7e666fe9f91d25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fb98c59b7d94d94af075f92b4bdd94a in term 1.
I20260811 18:40:26.362167 1809 consensus_queue.cc:237] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } }
I20260811 18:40:26.369022 1889 tablet_replica.cc:439] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a: TxnStatusTablet state changed. Reason: New leader 4fb98c59b7d94d94af075f92b4bdd94a. Latest consensus state: current_term: 1 leader_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } } }
I20260811 18:40:26.370853 1889 tablet_replica.cc:442] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:40:26.369422 923 catalog_manager.cc:5495] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4fb98c59b7d94d94af075f92b4bdd94a (127.30.185.131). New cstate: current_term: 1 leader_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } health_report { overall_health: HEALTHY } } }
I20260811 18:40:26.375331 1893 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 18:40:26.376008 1893 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 18:40:26.441819 1880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:26.424414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:40971 (local address 127.0.0.1:33266)
0811 18:40:26.425628 (+ 1214us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:26.425683 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:26.425999 (+ 316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:26.427203 (+ 1204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:26.427219 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:26.427246 (+ 27us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:40:26.427640 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:26.427683 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.430677 (+ 2994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.430685 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:26.433626 (+ 2941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:26.433664 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.436381 (+ 2717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.436391 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:26.437762 (+ 1371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:26.437937 (+ 175us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:40:26.440366 (+ 2429us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:40:26.440415 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 18:40:26.440553 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 18:40:26.440909 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":850}
I20260811 18:40:26.444023 1784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:26.425469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33266 (local address 127.30.185.131:40971)
0811 18:40:26.425879 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:26.425886 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:26.425933 (+ 47us) server_negotiation.cc:407] Connection header received
0811 18:40:26.426481 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:26.426499 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:26.426867 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:26.427134 (+ 267us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:40:26.427942 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.430414 (+ 2472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.433968 (+ 3554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.435739 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.436043 (+ 304us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:26.438452 (+ 2409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:40:26.439989 (+ 1537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:40:26.440262 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:26.443241 (+ 2979us) server_negotiation.cc:299] Negotiation successful
0811 18:40:26.443418 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175}
I20260811 18:40:26.456807 1416 raft_consensus.cc:1261] T a08252699ba541e9b312342cae4cf8d4 P 721f910c1ab04a5da8a0cfedfd0bf762 [term 1 FOLLOWER]: Refusing update from remote peer 4fb98c59b7d94d94af075f92b4bdd94a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:40:26.463029 1190 raft_consensus.cc:1261] T a08252699ba541e9b312342cae4cf8d4 P 4900b3066faf472f9b7e666fe9f91d25 [term 1 FOLLOWER]: Refusing update from remote peer 4fb98c59b7d94d94af075f92b4bdd94a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:40:26.462247 1889 consensus_queue.cc:1035] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260811 18:40:26.472564 1898 consensus_queue.cc:1035] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 18:40:26.498520 1812 tablet_replica.cc:439] T a08252699ba541e9b312342cae4cf8d4 P 4900b3066faf472f9b7e666fe9f91d25: TxnStatusTablet state changed. Reason: New leader 4fb98c59b7d94d94af075f92b4bdd94a. Latest consensus state: current_term: 1 leader_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } } }
I20260811 18:40:26.499353 1812 tablet_replica.cc:442] T a08252699ba541e9b312342cae4cf8d4 P 4900b3066faf472f9b7e666fe9f91d25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:40:26.504855 1810 tablet_replica.cc:439] T a08252699ba541e9b312342cae4cf8d4 P 721f910c1ab04a5da8a0cfedfd0bf762: TxnStatusTablet state changed. Reason: New leader 4fb98c59b7d94d94af075f92b4bdd94a. Latest consensus state: current_term: 1 leader_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } } }
I20260811 18:40:26.505599 1810 tablet_replica.cc:442] T a08252699ba541e9b312342cae4cf8d4 P 721f910c1ab04a5da8a0cfedfd0bf762: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:40:26.511039 1826 mvcc.cc:205] Tried to move back new op lower bound from 7317395973944135680 to 7317395973587275776. Current Snapshot: MvccSnapshot[applied={T|T < 7317395973944135680}]
I20260811 18:40:26.514190 1900 mvcc.cc:205] Tried to move back new op lower bound from 7317395973944135680 to 7317395973587275776. Current Snapshot: MvccSnapshot[applied={T|T < 7317395973944135680}]
I20260811 18:40:26.525899 1809 tablet_replica.cc:439] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } } }
I20260811 18:40:26.526790 1809 tablet_replica.cc:442] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:40:26.527973 1809 tablet_replica.cc:439] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } } }
I20260811 18:40:26.528604 1809 tablet_replica.cc:442] T a08252699ba541e9b312342cae4cf8d4 P 4fb98c59b7d94d94af075f92b4bdd94a: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:40:26.533208 1812 tablet_replica.cc:439] T a08252699ba541e9b312342cae4cf8d4 P 4900b3066faf472f9b7e666fe9f91d25: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } } }
I20260811 18:40:26.539811 1810 tablet_replica.cc:439] T a08252699ba541e9b312342cae4cf8d4 P 721f910c1ab04a5da8a0cfedfd0bf762: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4900b3066faf472f9b7e666fe9f91d25" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34513 } } peers { permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 } } peers { permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 } } }
I20260811 18:40:26.540393 1810 tablet_replica.cc:442] T a08252699ba541e9b312342cae4cf8d4 P 721f910c1ab04a5da8a0cfedfd0bf762: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:40:26.544862 1812 tablet_replica.cc:442] T a08252699ba541e9b312342cae4cf8d4 P 4900b3066faf472f9b7e666fe9f91d25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:40:26.568310 1792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:26.551941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53538 (local address 127.30.185.129:34513)
0811 18:40:26.552255 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:26.552274 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:26.552560 (+ 286us) server_negotiation.cc:407] Connection header received
0811 18:40:26.553267 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:26.553275 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:26.553591 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:26.553923 (+ 332us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:40:26.555868 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.558643 (+ 2775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.561656 (+ 3013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.563012 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.563223 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:26.564658 (+ 1435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:40:26.566022 (+ 1364us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:40:26.566307 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:26.567547 (+ 1240us) server_negotiation.cc:299] Negotiation successful
0811 18:40:26.567752 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 18:40:26.591022 1812 consensus_queue.cc:1035] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "721f910c1ab04a5da8a0cfedfd0bf762" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:40:26.620226 1812 consensus_queue.cc:1035] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fb98c59b7d94d94af075f92b4bdd94a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:40:26.639353 1784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:26.614616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:51031 (local address 127.30.185.131:40971)
0811 18:40:26.614845 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:26.614852 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:26.622661 (+ 7809us) server_negotiation.cc:407] Connection header received
0811 18:40:26.622986 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:26.622994 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:26.623100 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:26.623172 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:40:26.624546 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.625858 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.635244 (+ 9386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.637239 (+ 1995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.637396 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:26.638154 (+ 758us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:40:26.638303 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:40:26.638376 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:26.638633 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 18:40:26.638762 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:40:26.640484 1926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:26.614134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:40971 (local address 127.30.185.129:51031)
0811 18:40:26.622523 (+ 8389us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:26.622557 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:26.622807 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:26.623414 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:26.623422 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:26.623461 (+ 39us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:40:26.624325 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:26.624343 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.626112 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.626120 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:26.630731 (+ 4611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:26.630755 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.637576 (+ 6821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.637586 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:26.637810 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:26.637901 (+ 91us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:40:26.638473 (+ 572us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:40:26.638486 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 18:40:26.639638 (+ 1152us) client_negotiation.cc:240] Negotiation successful
0811 18:40:26.639821 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8136,"thread_start_us":11733,"threads_started":1}
I20260811 18:40:26.711454 1792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:26.698042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:35047 (local address 127.30.185.129:34513)
0811 18:40:26.698253 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:26.698260 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:26.701836 (+ 3576us) server_negotiation.cc:407] Connection header received
0811 18:40:26.702155 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:26.702162 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:26.702268 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:26.702332 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:40:26.703830 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.705002 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.707861 (+ 2859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.709121 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.709291 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:26.710201 (+ 910us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:40:26.710349 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:40:26.710496 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:26.710711 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 18:40:26.710805 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 18:40:26.712594 1938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:26.697566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:34513 (local address 127.30.185.131:35047)
0811 18:40:26.701715 (+ 4149us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:26.701742 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:26.701964 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:26.702528 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:26.702535 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:26.702549 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:40:26.703561 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:26.703576 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.705162 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.705187 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:26.707663 (+ 2476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:26.707684 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.709539 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.709546 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:26.709826 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:26.709946 (+ 120us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:40:26.710501 (+ 555us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:40:26.710515 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 18:40:26.711839 (+ 1324us) client_negotiation.cc:240] Negotiation successful
0811 18:40:26.712030 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3927,"thread_start_us":871,"threads_started":1}
I20260811 18:40:26.785367 926 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:40:26.785660 926 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:40:26.789780 926 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c84a3e8a34474f25ab1592eaae79f988: Sending DeleteTablet for 3 replicas of tablet 1a180b8209df410ca3fc1c57c7725ed1
I20260811 18:40:26.791618 1157 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a180b8209df410ca3fc1c57c7725ed1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:26 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:53532
I20260811 18:40:26.792138 1385 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a180b8209df410ca3fc1c57c7725ed1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:26 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:45098
I20260811 18:40:26.792908 1621 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a180b8209df410ca3fc1c57c7725ed1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:26 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:33258
I20260811 18:40:26.838129 1947 tablet_replica.cc:330] T 1a180b8209df410ca3fc1c57c7725ed1 P 4fb98c59b7d94d94af075f92b4bdd94a: stopping tablet replica
I20260811 18:40:26.838541 1947 raft_consensus.cc:2229] T 1a180b8209df410ca3fc1c57c7725ed1 P 4fb98c59b7d94d94af075f92b4bdd94a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:40:26.839072 1947 raft_consensus.cc:2258] T 1a180b8209df410ca3fc1c57c7725ed1 P 4fb98c59b7d94d94af075f92b4bdd94a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:40:26.840610 1947 ts_tablet_manager.cc:1905] T 1a180b8209df410ca3fc1c57c7725ed1 P 4fb98c59b7d94d94af075f92b4bdd94a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:40:26.844863 1944 tablet_replica.cc:330] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25: stopping tablet replica
I20260811 18:40:26.845094 1944 raft_consensus.cc:2229] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:40:26.846089 1944 raft_consensus.cc:2258] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:40:26.846400 1945 tablet_replica.cc:330] T 1a180b8209df410ca3fc1c57c7725ed1 P 721f910c1ab04a5da8a0cfedfd0bf762: stopping tablet replica
I20260811 18:40:26.846623 1945 raft_consensus.cc:2229] T 1a180b8209df410ca3fc1c57c7725ed1 P 721f910c1ab04a5da8a0cfedfd0bf762 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:40:26.846879 1945 raft_consensus.cc:2258] T 1a180b8209df410ca3fc1c57c7725ed1 P 721f910c1ab04a5da8a0cfedfd0bf762 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:40:26.848098 1945 ts_tablet_manager.cc:1905] T 1a180b8209df410ca3fc1c57c7725ed1 P 721f910c1ab04a5da8a0cfedfd0bf762: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:40:26.853722 1944 ts_tablet_manager.cc:1905] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:40:26.866353 1945 ts_tablet_manager.cc:1918] T 1a180b8209df410ca3fc1c57c7725ed1 P 721f910c1ab04a5da8a0cfedfd0bf762: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:40:26.866544 1945 log.cc:1196] T 1a180b8209df410ca3fc1c57c7725ed1 P 721f910c1ab04a5da8a0cfedfd0bf762: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/1a180b8209df410ca3fc1c57c7725ed1
I20260811 18:40:26.867049 1945 ts_tablet_manager.cc:1939] T 1a180b8209df410ca3fc1c57c7725ed1 P 721f910c1ab04a5da8a0cfedfd0bf762: Deleting consensus metadata
Aug 11 18:40:26 dist-test-slave-z8x9 krb5kdc[855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473625, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.185.129@KRBTEST.COM
I20260811 18:40:26.874428 897 catalog_manager.cc:4841] TS 721f910c1ab04a5da8a0cfedfd0bf762 (127.30.185.130:33361): tablet 1a180b8209df410ca3fc1c57c7725ed1 (table test-table [id=582589b95bb143df8501761d4eb3bac7]) successfully deleted
I20260811 18:40:26.877228 1944 ts_tablet_manager.cc:1918] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:40:26.877424 1944 log.cc:1196] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-0/wal/wals/1a180b8209df410ca3fc1c57c7725ed1
I20260811 18:40:26.869088 1947 ts_tablet_manager.cc:1918] T 1a180b8209df410ca3fc1c57c7725ed1 P 4fb98c59b7d94d94af075f92b4bdd94a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:40:26.878134 1947 log.cc:1196] T 1a180b8209df410ca3fc1c57c7725ed1 P 4fb98c59b7d94d94af075f92b4bdd94a: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/1a180b8209df410ca3fc1c57c7725ed1
I20260811 18:40:26.878664 1947 ts_tablet_manager.cc:1939] T 1a180b8209df410ca3fc1c57c7725ed1 P 4fb98c59b7d94d94af075f92b4bdd94a: Deleting consensus metadata
I20260811 18:40:26.879760 1944 ts_tablet_manager.cc:1939] T 1a180b8209df410ca3fc1c57c7725ed1 P 4900b3066faf472f9b7e666fe9f91d25: Deleting consensus metadata
I20260811 18:40:26.880899 897 catalog_manager.cc:4841] TS 4fb98c59b7d94d94af075f92b4bdd94a (127.30.185.131:40971): tablet 1a180b8209df410ca3fc1c57c7725ed1 (table test-table [id=582589b95bb143df8501761d4eb3bac7]) successfully deleted
I20260811 18:40:26.881438 895 catalog_manager.cc:4841] TS 4900b3066faf472f9b7e666fe9f91d25 (127.30.185.129:34513): tablet 1a180b8209df410ca3fc1c57c7725ed1 (table test-table [id=582589b95bb143df8501761d4eb3bac7]) successfully deleted
I20260811 18:40:26.886204 1792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:26.844835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53542 (local address 127.30.185.129:34513)
0811 18:40:26.845049 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:26.845056 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:26.857151 (+ 12095us) server_negotiation.cc:407] Connection header received
0811 18:40:26.857499 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:26.857506 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:26.857589 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:26.857663 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:26.861222 (+ 3559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.862694 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.867331 (+ 4637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.868133 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.868238 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:26.876346 (+ 8108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:26.876365 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:26.876385 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:26.876436 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:26.882377 (+ 5941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:26.883333 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:26.883345 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:26.883349 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:26.883545 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:26.884115 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:26.884124 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:26.884127 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:26.884590 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:26.884949 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:26.885345 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 18:40:26.885580 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 18:40:26.887087 1265 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53542
I20260811 18:40:26.925215 1961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:26.894632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39250 (local address 127.30.185.190:44247)
0811 18:40:26.905550 (+ 10918us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:26.905556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:26.905580 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:40:26.905650 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:26.905656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:26.905737 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:26.905856 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:26.908064 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.909435 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.911656 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.912657 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.912792 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:26.917225 (+ 4433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:26.917233 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:26.917239 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:26.917301 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:26.921260 (+ 3959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:26.922239 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:26.922245 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:26.922248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:26.922464 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:26.922989 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:26.922994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:26.922999 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:26.923515 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:26.923825 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:26.924213 (+ 388us) server_negotiation.cc:299] Negotiation successful
0811 18:40:26.924500 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10705,"thread_start_us":10549,"threads_started":1}
W20260811 18:40:26.925850 926 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39250
I20260811 18:40:26.928216 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 1042
I20260811 18:40:26.952781 1792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:26.930400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53548 (local address 127.30.185.129:34513)
0811 18:40:26.930602 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:26.930608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:26.934104 (+ 3496us) server_negotiation.cc:407] Connection header received
0811 18:40:26.934304 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:26.934311 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:26.934441 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:26.934620 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:26.936325 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.937641 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.939600 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.940409 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:26.940596 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:26.944190 (+ 3594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:26.944199 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:26.944204 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:26.944276 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:26.948960 (+ 4684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:26.950070 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:26.950077 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:26.950080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:26.950320 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:26.950978 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:26.950984 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:26.950990 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:26.951394 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:26.951699 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:26.952048 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 18:40:26.952343 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 18:40:26.953697 1265 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53548
W20260811 18:40:26.954454 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 1042: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:40:26.954818 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 1042
I20260811 18:40:26.994288 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 1277
Aug 11 18:40:27 dist-test-slave-z8x9 krb5kdc[855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473625, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.185.130@KRBTEST.COM
I20260811 18:40:27.016821 1963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:26.995154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45118 (local address 127.30.185.130:33361)
0811 18:40:26.996184 (+ 1030us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:26.996191 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:26.996211 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:40:26.996288 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:26.996293 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:26.996376 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:26.996513 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:26.998248 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:26.999463 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:27.001364 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:27.002116 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:27.002261 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:27.008056 (+ 5795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:27.008075 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:27.008096 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:27.008144 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:27.012944 (+ 4800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:27.014139 (+ 1195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:27.014150 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:27.014153 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:27.014328 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:27.014814 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:27.014821 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:27.014826 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:27.015307 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:27.015695 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:27.015909 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 18:40:27.016180 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":907,"thread_start_us":651,"threads_started":1}
W20260811 18:40:27.017674 1487 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45118
W20260811 18:40:27.018481 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 1277: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:40:27.018676 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 1277
I20260811 18:40:27.060828 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 1499
Aug 11 18:40:27 dist-test-slave-z8x9 krb5kdc[855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473625, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.185.131@KRBTEST.COM
I20260811 18:40:27.079708 1784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:27.061701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33280 (local address 127.30.185.131:40971)
0811 18:40:27.061880 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:27.061885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:27.061976 (+ 91us) server_negotiation.cc:407] Connection header received
0811 18:40:27.062185 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:27.062189 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:27.062250 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:27.062339 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:27.064185 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:27.064948 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:27.066855 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:27.067290 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:27.067389 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:27.072734 (+ 5345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:27.072745 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:27.072761 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:27.072801 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:27.076470 (+ 3669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:27.077324 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:27.077332 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:27.077334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:27.077501 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:27.078004 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:27.078008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:27.078010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:27.078391 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:27.078782 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:27.079051 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 18:40:27.079315 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 18:40:27.080533 1743 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33280
W20260811 18:40:27.081291 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 1499: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:40:27.081489 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 1499
I20260811 18:40:27.113423 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 872
I20260811 18:40:27.115031 993 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:27.234923 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 872
2026-08-11T18:40:27Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5597 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:40:27 dist-test-slave-z8x9 krb5kdc[1968](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:40:27 dist-test-slave-z8x9 krb5kdc[1968](info): set up 2 sockets
Aug 11 18:40:27 dist-test-slave-z8x9 krb5kdc[1968](info): commencing operation
krb5kdc: starting...
W20260811 18:40:28.664435 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.373s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:40:28 dist-test-slave-z8x9 krb5kdc[1968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:40:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:40:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:40:28.917758 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:42283
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:43309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:42283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:29.257063 1992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:29.257524 1992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:29.257699 1992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:29.265051 1992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:40:29.265230 1992 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 18:40:29.265329 1992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:29.265439 1992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:29.265530 1992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:40:29.265627 1992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:40:29.275753 1992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:43309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:42283
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:42283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:29.277109 1992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:29.278510 1992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:29.290489 2002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:29.290500 1999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:29.290827 1998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:29.291834 1992 server_base.cc:1034] running on GCE node
I20260811 18:40:29.292677 1992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:29.294179 1992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:29.295424 1992 hybrid_clock.cc:648] HybridClock initialized: now 1786473629295349 us; error 34 us; skew 500 ppm
I20260811 18:40:29.295784 1992 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:29 dist-test-slave-z8x9 krb5kdc[1968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473629, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:29.299283 1992 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:40:29.300990 1992 webserver.cc:460] Webserver started at http://127.30.185.190:45421/ using document root <none> and password file <none>
I20260811 18:40:29.301810 1992 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:29.301906 1992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:29.302271 1992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:29.305629 1992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "fb7753544bb949f6b8798dcaa4467435"
format_stamp: "Formatted at 2026-08-11 18:40:29 on dist-test-slave-z8x9"
server_key: "e6578fbb663078dd8739ca42bdb9a1c9"
server_key_iv: "d097629f4f104821ea98daa83ee91db1"
server_key_version: "encryptionkey@0"
I20260811 18:40:29.306478 1992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "fb7753544bb949f6b8798dcaa4467435"
format_stamp: "Formatted at 2026-08-11 18:40:29 on dist-test-slave-z8x9"
server_key: "e6578fbb663078dd8739ca42bdb9a1c9"
server_key_iv: "d097629f4f104821ea98daa83ee91db1"
server_key_version: "encryptionkey@0"
I20260811 18:40:29.311990 1992 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 18:40:29.315644 2014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:29.316946 1992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 18:40:29.317152 1992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "fb7753544bb949f6b8798dcaa4467435"
format_stamp: "Formatted at 2026-08-11 18:40:29 on dist-test-slave-z8x9"
server_key: "e6578fbb663078dd8739ca42bdb9a1c9"
server_key_iv: "d097629f4f104821ea98daa83ee91db1"
server_key_version: "encryptionkey@0"
I20260811 18:40:29.317469 1992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:29.345700 1992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:29.350461 1992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:29.350785 1992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:29.374722 1992 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:42283
I20260811 18:40:29.374747 2130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:42283 every 8 connection(s)
I20260811 18:40:29.376178 1992 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:40:29.380295 2131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:29.382287 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 1992
I20260811 18:40:29.382561 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:40:29.383515 31462 external_mini_cluster.cc:1428] Setting key cc7da5914c1a52f7ad13e06897938be3
I20260811 18:40:29.395350 2131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435: Bootstrap starting.
I20260811 18:40:29.400125 2131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435: Neither blocks nor log segments found. Creating new log.
Aug 11 18:40:29 dist-test-slave-z8x9 krb5kdc[1968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:29.401487 2131 log.cc:824] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:29.405659 2131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435: No bootstrap required, opened a new log
I20260811 18:40:29.411783 2136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:29.386026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55552 (local address 127.30.185.190:42283)
0811 18:40:29.386883 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:29.386895 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:29.386944 (+ 49us) server_negotiation.cc:407] Connection header received
0811 18:40:29.388728 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:29.388753 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:29.389162 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:29.389574 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:29.391765 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:29.393323 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:29.395328 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:29.396101 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:29.396249 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:29.401686 (+ 5437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:29.401701 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:29.401723 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:29.401778 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:29.406467 (+ 4689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:29.407241 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:29.407298 (+ 57us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:29.407308 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:29.407557 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:29.407945 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:29.407952 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:29.407960 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:29.408821 (+ 861us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:29.409154 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:29.409674 (+ 520us) server_negotiation.cc:299] Negotiation successful
0811 18:40:29.410212 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":410,"threads_started":1}
I20260811 18:40:29.416956 2131 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7753544bb949f6b8798dcaa4467435" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 42283 } }
I20260811 18:40:29.417474 2131 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:29.417565 2131 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb7753544bb949f6b8798dcaa4467435, State: Initialized, Role: FOLLOWER
I20260811 18:40:29.418220 2131 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7753544bb949f6b8798dcaa4467435" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 42283 } }
I20260811 18:40:29.418460 2131 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:40:29.418591 2131 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:40:29.418787 2131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:29.421506 2131 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7753544bb949f6b8798dcaa4467435" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 42283 } }
I20260811 18:40:29.422169 2131 leader_election.cc:302] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb7753544bb949f6b8798dcaa4467435; no voters:
I20260811 18:40:29.422820 2131 leader_election.cc:288] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:40:29.423043 2139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:40:29.423821 2139 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435 [term 1 LEADER]: Becoming Leader. State: Replica: fb7753544bb949f6b8798dcaa4467435, State: Running, Role: LEADER
I20260811 18:40:29.424454 2139 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7753544bb949f6b8798dcaa4467435" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 42283 } }
I20260811 18:40:29.425415 2131 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:40:29.428119 2141 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb7753544bb949f6b8798dcaa4467435. Latest consensus state: current_term: 1 leader_uuid: "fb7753544bb949f6b8798dcaa4467435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7753544bb949f6b8798dcaa4467435" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 42283 } } }
I20260811 18:40:29.428712 2141 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435 [sys.catalog]: This master's current role is: LEADER
I20260811 18:40:29.432502 2140 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb7753544bb949f6b8798dcaa4467435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7753544bb949f6b8798dcaa4467435" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 42283 } } }
I20260811 18:40:29.432889 2140 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435 [sys.catalog]: This master's current role is: LEADER
I20260811 18:40:29.437870 2150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:40:29.443395 2150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:40:29.451706 2150 catalog_manager.cc:1347] Generated new cluster ID: a829b217087247d083f5639aa15e987c
I20260811 18:40:29.451848 2150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:40:29.468140 2150 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:40:29.470337 2150 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:40:29.482477 2150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435: Generated new TSK 0
I20260811 18:40:29.483645 2150 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.185.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
I20260811 18:40:29.551209 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:42283
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:43309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:29.893705 2171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:29.894069 2171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:29.894260 2171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:29.901556 2171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:29.901729 2171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:29.901885 2171 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 18:40:29.901973 2171 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:40:29.902099 2171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:40:29.912210 2171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:43309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.185.190:42283
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:29.913724 2171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:29.915160 2171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:29.929343 2179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:29.929389 2181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:29.929921 2185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:29.930183 2171 server_base.cc:1034] running on GCE node
I20260811 18:40:29.930784 2171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:29.931777 2171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:29.932991 2171 hybrid_clock.cc:648] HybridClock initialized: now 1786473629932939 us; error 40 us; skew 500 ppm
I20260811 18:40:29.933372 2171 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:29 dist-test-slave-z8x9 krb5kdc[1968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473629, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:29.937004 2171 init.cc:377] Logged in from keytab as kudu/127.30.185.129@KRBTEST.COM (short username kudu)
I20260811 18:40:29.938627 2171 webserver.cc:460] Webserver started at http://127.30.185.129:37531/ using document root <none> and password file <none>
I20260811 18:40:29.939298 2171 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:29.939411 2171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:29.939684 2171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:29.942652 2171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "7209845a058349c6a108c7fa04c5b580"
format_stamp: "Formatted at 2026-08-11 18:40:29 on dist-test-slave-z8x9"
server_key: "9e46ef95b436014dd62dedb83fb0b5a7"
server_key_iv: "739236a4d746ff7d1b37be82f847c9da"
server_key_version: "encryptionkey@0"
I20260811 18:40:29.943395 2171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "7209845a058349c6a108c7fa04c5b580"
format_stamp: "Formatted at 2026-08-11 18:40:29 on dist-test-slave-z8x9"
server_key: "9e46ef95b436014dd62dedb83fb0b5a7"
server_key_iv: "739236a4d746ff7d1b37be82f847c9da"
server_key_version: "encryptionkey@0"
I20260811 18:40:29.949342 2171 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 18:40:29.952858 2196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:29.954319 2171 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:40:29.954515 2171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "7209845a058349c6a108c7fa04c5b580"
format_stamp: "Formatted at 2026-08-11 18:40:29 on dist-test-slave-z8x9"
server_key: "9e46ef95b436014dd62dedb83fb0b5a7"
server_key_iv: "739236a4d746ff7d1b37be82f847c9da"
server_key_version: "encryptionkey@0"
I20260811 18:40:29.954744 2171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:29.970301 2171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:29.974503 2171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:29.974761 2171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:29.977661 2171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:29.977775 2171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:29.977955 2171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:29.978065 2171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:40:29 dist-test-slave-z8x9 krb5kdc[1968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473629, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:30.009713 2218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:29.978286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:56021 (local address 127.30.185.190:42283)
0811 18:40:29.978993 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:29.979000 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:29.980133 (+ 1133us) server_negotiation.cc:407] Connection header received
0811 18:40:29.983319 (+ 3186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:29.983325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:29.983418 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:29.983531 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:29.986029 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:29.987358 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:29.990151 (+ 2793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:29.991105 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:29.991276 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:29.997805 (+ 6529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:29.997815 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:29.997822 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:29.997873 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:30.001839 (+ 3966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:30.002998 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:30.003006 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:30.003013 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:30.003204 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:30.003989 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:30.003996 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:30.004002 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:30.004467 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:30.004665 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:30.008997 (+ 4332us) server_negotiation.cc:299] Negotiation successful
0811 18:40:30.009367 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":449,"threads_started":1}
I20260811 18:40:30.010233 2220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:29.978961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:42283 (local address 127.30.185.129:56021)
0811 18:40:29.979926 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:29.979975 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:29.982921 (+ 2946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:29.983933 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:29.983947 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:29.984807 (+ 860us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:29.985789 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:29.985829 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:29.987567 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:29.987572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:29.989969 (+ 2397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:29.989982 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:29.991424 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:29.991439 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:29.991855 (+ 416us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:29.992831 (+ 976us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:29.992874 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:29.997445 (+ 4571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:30.002090 (+ 4645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:30.002106 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:30.002134 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:30.002837 (+ 703us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:30.003402 (+ 565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:30.003408 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:30.003412 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:30.003801 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:30.004667 (+ 866us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:30.004681 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:30.005426 (+ 745us) client_negotiation.cc:769] Sending connection context
0811 18:40:30.005842 (+ 416us) client_negotiation.cc:240] Negotiation successful
0811 18:40:30.006700 (+ 858us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":389,"threads_started":1}
I20260811 18:40:30.033812 2218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:30.017042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:39655 (local address 127.30.185.190:42283)
0811 18:40:30.017258 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:30.017263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:30.018116 (+ 853us) server_negotiation.cc:407] Connection header received
0811 18:40:30.018364 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:30.018373 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:30.018485 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:30.018600 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:30.019557 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:30.020850 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:30.022960 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:30.023532 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:30.023670 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:30.026663 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:30.026671 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:30.026678 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:30.026724 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:30.030599 (+ 3875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:30.031411 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:30.031417 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:30.031420 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:30.031569 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:30.032053 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:30.032057 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:30.032061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:30.032419 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:30.032583 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:30.033134 (+ 551us) server_negotiation.cc:299] Negotiation successful
0811 18:40:30.033419 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 18:40:30.034022 2346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:30.017029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:42283 (local address 127.30.185.129:39655)
0811 18:40:30.017975 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:30.017998 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:30.018173 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:30.018652 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:30.018657 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:30.019103 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:30.019403 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:30.019412 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:30.021055 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:30.021060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:30.022813 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:30.022826 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:30.023761 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:30.023768 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:30.023944 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:30.024801 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:30.024838 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:30.026368 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:30.030802 (+ 4434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:30.030810 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:30.030821 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:30.031265 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:30.031691 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:30.031698 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:30.031702 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:30.031897 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:30.032545 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:30.032550 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:30.032823 (+ 273us) client_negotiation.cc:769] Sending connection context
0811 18:40:30.033013 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 18:40:30.033415 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"thread_start_us":567,"threads_started":1}
I20260811 18:40:30.034672 2171 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:45641
I20260811 18:40:30.034782 2414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:45641 every 8 connection(s)
I20260811 18:40:30.036319 2171 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:40:30.036572 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 2171
I20260811 18:40:30.036757 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:40:30.037662 31462 external_mini_cluster.cc:1428] Setting key b46cc5bf9e1c2b67fc07c792159a9f8d
I20260811 18:40:30.039198 2415 heartbeater.cc:347] Connected to a master server at 127.30.185.190:42283
I20260811 18:40:30.039438 2415 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:30.040292 2415 heartbeater.cc:511] Master 127.30.185.190:42283 requested a full tablet report, sending...
I20260811 18:40:30.042493 2042 ts_manager.cc:194] Registered new tserver with Master: 7209845a058349c6a108c7fa04c5b580 (127.30.185.129:45641)
I20260811 18:40:30.043987 2042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:56021
WARNING: no policy specified for kudu/127.30.185.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
I20260811 18:40:30.090487 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:42283
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:43309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:30.431527 2424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:30.431815 2424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:30.431980 2424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:30.439343 2424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:30.439447 2424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:30.439591 2424 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 18:40:30.439683 2424 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:40:30.439807 2424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:40:30.449297 2424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:43309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.185.190:42283
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:30.450742 2424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:30.452023 2424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:30.465349 2430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:30.465700 2434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:30.466054 2431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:30.466768 2424 server_base.cc:1034] running on GCE node
I20260811 18:40:30.467481 2424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:30.468511 2424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:30.469769 2424 hybrid_clock.cc:648] HybridClock initialized: now 1786473630469723 us; error 50 us; skew 500 ppm
I20260811 18:40:30.470136 2424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:30 dist-test-slave-z8x9 krb5kdc[1968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473630, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:30.473702 2424 init.cc:377] Logged in from keytab as kudu/127.30.185.130@KRBTEST.COM (short username kudu)
I20260811 18:40:30.475317 2424 webserver.cc:460] Webserver started at http://127.30.185.130:36123/ using document root <none> and password file <none>
I20260811 18:40:30.476081 2424 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:30.476250 2424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:30.476612 2424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:30.479907 2424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "a1e13f628490435d85fa802f60e2d0f7"
format_stamp: "Formatted at 2026-08-11 18:40:30 on dist-test-slave-z8x9"
server_key: "47857268090f1920ea022c540f3c7a41"
server_key_iv: "4c4a7d9209dc44de9ccfd01b67b8a7a9"
server_key_version: "encryptionkey@0"
I20260811 18:40:30.480751 2424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "a1e13f628490435d85fa802f60e2d0f7"
format_stamp: "Formatted at 2026-08-11 18:40:30 on dist-test-slave-z8x9"
server_key: "47857268090f1920ea022c540f3c7a41"
server_key_iv: "4c4a7d9209dc44de9ccfd01b67b8a7a9"
server_key_version: "encryptionkey@0"
I20260811 18:40:30.486474 2424 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 18:40:30.490214 2449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:30.491628 2424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:40:30.491829 2424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "a1e13f628490435d85fa802f60e2d0f7"
format_stamp: "Formatted at 2026-08-11 18:40:30 on dist-test-slave-z8x9"
server_key: "47857268090f1920ea022c540f3c7a41"
server_key_iv: "4c4a7d9209dc44de9ccfd01b67b8a7a9"
server_key_version: "encryptionkey@0"
I20260811 18:40:30.492046 2424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:30.532449 2424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:30.537810 2424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:30.538141 2424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:30.541024 2424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:30.541199 2424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:30.541395 2424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:30.541509 2424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:40:30 dist-test-slave-z8x9 krb5kdc[1968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473630, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:30.567384 2462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:30.541830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:52075 (local address 127.30.185.190:42283)
0811 18:40:30.542539 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:30.542546 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:30.543635 (+ 1089us) server_negotiation.cc:407] Connection header received
0811 18:40:30.547099 (+ 3464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:30.547105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:30.547198 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:30.547354 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:30.549651 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:30.550685 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:30.553399 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:30.554391 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:30.554591 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:30.558965 (+ 4374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:30.558975 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:30.558984 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:30.559033 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:30.563559 (+ 4526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:30.564517 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:30.564523 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:30.564529 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:30.564665 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:30.565294 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:30.565299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:30.565301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:30.565703 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:30.565870 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:30.566745 (+ 875us) server_negotiation.cc:299] Negotiation successful
0811 18:40:30.566997 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":421,"threads_started":1}
I20260811 18:40:30.568436 2463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:30.542510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:42283 (local address 127.30.185.130:52075)
0811 18:40:30.543422 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:30.543486 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:30.546700 (+ 3214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:30.547673 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:30.547691 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:30.548535 (+ 844us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:30.549456 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:30.549479 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:30.550859 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:30.550867 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:30.553215 (+ 2348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:30.553238 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:30.554643 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:30.554649 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:30.554832 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:30.555644 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:30.555672 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:30.558650 (+ 2978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:30.563757 (+ 5107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:30.563768 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:30.563792 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:30.564371 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:30.564792 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:30.564806 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:30.564812 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:30.565140 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:30.565943 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:30.565949 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:30.566339 (+ 390us) client_negotiation.cc:769] Sending connection context
0811 18:40:30.566662 (+ 323us) client_negotiation.cc:240] Negotiation successful
0811 18:40:30.567195 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":370,"threads_started":1}
I20260811 18:40:30.593529 2462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:30.577490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:55459 (local address 127.30.185.190:42283)
0811 18:40:30.577756 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:30.577765 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:30.578433 (+ 668us) server_negotiation.cc:407] Connection header received
0811 18:40:30.578753 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:30.578758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:30.578827 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:30.578918 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:30.580585 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:30.581391 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:30.583536 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:30.584164 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:30.584321 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:30.587181 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:30.587190 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:30.587199 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:30.587232 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:30.590457 (+ 3225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:30.591488 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:30.591492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:30.591494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:30.591610 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:30.591993 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:30.591998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:30.592002 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:30.592342 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:30.592503 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:30.592912 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 18:40:30.593150 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 18:40:30.593708 2595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:30.577488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:42283 (local address 127.30.185.130:55459)
0811 18:40:30.578292 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:30.578319 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:30.578551 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:30.578994 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:30.579003 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:30.579868 (+ 865us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:30.580392 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:30.580409 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:30.581543 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:30.581551 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:30.583373 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:30.583390 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:30.584611 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:30.584622 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:30.584833 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:30.585659 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:30.585676 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:30.586951 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:30.590640 (+ 3689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:30.590649 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:30.590654 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:30.591306 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:30.591731 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:30.591736 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:30.591739 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:30.591881 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:30.592466 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:30.592471 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:30.592661 (+ 190us) client_negotiation.cc:769] Sending connection context
0811 18:40:30.592815 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 18:40:30.593157 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":402,"threads_started":1}
I20260811 18:40:30.602170 2424 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:44815
I20260811 18:40:30.602206 2660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:44815 every 8 connection(s)
I20260811 18:40:30.603528 2424 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:40:30.604959 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 2424
I20260811 18:40:30.605240 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:40:30.605880 31462 external_mini_cluster.cc:1428] Setting key 6daf58422325330ac028067e2516506b
I20260811 18:40:30.605912 2662 heartbeater.cc:347] Connected to a master server at 127.30.185.190:42283
I20260811 18:40:30.606146 2662 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:30.606976 2662 heartbeater.cc:511] Master 127.30.185.190:42283 requested a full tablet report, sending...
I20260811 18:40:30.608385 2042 ts_manager.cc:194] Registered new tserver with Master: a1e13f628490435d85fa802f60e2d0f7 (127.30.185.130:44815)
I20260811 18:40:30.609344 2042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:52075
WARNING: no policy specified for kudu/127.30.185.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
I20260811 18:40:30.660985 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:42283
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:43309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:31.000084 2666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:31.000432 2666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:31.000618 2666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:31.008176 2666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:31.008375 2666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:31.008530 2666 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 18:40:31.008620 2666 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:40:31.008760 2666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:40:31.018828 2666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:43309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.185.190:42283
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:31.020179 2666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:31.021704 2666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:31.035346 2673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:31.035466 2676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:31.035501 2680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:31.036932 2666 server_base.cc:1034] running on GCE node
I20260811 18:40:31.037663 2666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:31.038637 2666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:31.039868 2666 hybrid_clock.cc:648] HybridClock initialized: now 1786473631039812 us; error 48 us; skew 500 ppm
I20260811 18:40:31.040180 2666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:31 dist-test-slave-z8x9 krb5kdc[1968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473631, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:31.043772 2666 init.cc:377] Logged in from keytab as kudu/127.30.185.131@KRBTEST.COM (short username kudu)
I20260811 18:40:31.045372 2666 webserver.cc:460] Webserver started at http://127.30.185.131:36301/ using document root <none> and password file <none>
I20260811 18:40:31.046077 2666 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:31.046226 2666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:31.046556 2666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:31.046790 2415 heartbeater.cc:503] Master 127.30.185.190:42283 was elected leader, sending a full tablet report...
I20260811 18:40:31.049603 2666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "e4133e575b754c4fb437792fca3afd09"
format_stamp: "Formatted at 2026-08-11 18:40:31 on dist-test-slave-z8x9"
server_key: "219a9547865f7d40f8eff23615a85f10"
server_key_iv: "23491829f478a7553ee7d3ffee9fc40c"
server_key_version: "encryptionkey@0"
I20260811 18:40:31.050383 2666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "e4133e575b754c4fb437792fca3afd09"
format_stamp: "Formatted at 2026-08-11 18:40:31 on dist-test-slave-z8x9"
server_key: "219a9547865f7d40f8eff23615a85f10"
server_key_iv: "23491829f478a7553ee7d3ffee9fc40c"
server_key_version: "encryptionkey@0"
I20260811 18:40:31.055475 2666 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260811 18:40:31.059152 2689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:31.060631 2666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:40:31.060811 2666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "e4133e575b754c4fb437792fca3afd09"
format_stamp: "Formatted at 2026-08-11 18:40:31 on dist-test-slave-z8x9"
server_key: "219a9547865f7d40f8eff23615a85f10"
server_key_iv: "23491829f478a7553ee7d3ffee9fc40c"
server_key_version: "encryptionkey@0"
I20260811 18:40:31.061071 2666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:31.080476 2666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:31.084630 2666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:31.084942 2666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:31.087656 2666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:31.087813 2666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:31.087958 2666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:31.088057 2666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:40:31 dist-test-slave-z8x9 krb5kdc[1968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473631, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:31.118223 2462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.088791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:35835 (local address 127.30.185.190:42283)
0811 18:40:31.089018 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:31.089025 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:31.090491 (+ 1466us) server_negotiation.cc:407] Connection header received
0811 18:40:31.093105 (+ 2614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:31.093116 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:31.093304 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:31.093436 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:31.095821 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.097231 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.100764 (+ 3533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.101660 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.101821 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.108882 (+ 7061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:31.108892 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:31.108899 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:31.108950 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:31.113678 (+ 4728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:31.114676 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:31.114685 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:31.114690 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:31.114882 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:31.115764 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:31.115770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:31.115773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:31.116129 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:31.116288 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:31.117513 (+ 1225us) server_negotiation.cc:299] Negotiation successful
0811 18:40:31.117835 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 18:40:31.120043 2709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.089316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:42283 (local address 127.30.185.131:35835)
0811 18:40:31.090281 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:31.090337 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:31.092610 (+ 2273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:31.093782 (+ 1172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:31.093795 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:31.094566 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:31.095558 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.095599 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.097540 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.097551 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.100506 (+ 2955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.100531 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.102336 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.102348 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.102727 (+ 379us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.103799 (+ 1072us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:31.103841 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:31.108542 (+ 4701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:31.113890 (+ 5348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:31.113900 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:31.113930 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:31.114535 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:31.115068 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:31.115073 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:31.115081 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:31.115588 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:31.116292 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:31.116302 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:31.116923 (+ 621us) client_negotiation.cc:769] Sending connection context
0811 18:40:31.117369 (+ 446us) client_negotiation.cc:240] Negotiation successful
0811 18:40:31.118030 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":410,"threads_started":1}
I20260811 18:40:31.151268 2878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.130749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:42283 (local address 127.30.185.131:39851)
0811 18:40:31.131458 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:31.131474 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:31.131649 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:31.132144 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:31.132150 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:31.133231 (+ 1081us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:31.133840 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.133866 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.135221 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.135229 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.138127 (+ 2898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.138150 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.139391 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.139396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.139545 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.140665 (+ 1120us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:31.140706 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:31.142476 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:31.147708 (+ 5232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:31.147717 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:31.147721 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:31.148266 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:31.148709 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:31.148715 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:31.148719 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:31.148914 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:31.149517 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:31.149522 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:31.149829 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 18:40:31.150052 (+ 223us) client_negotiation.cc:240] Negotiation successful
0811 18:40:31.150577 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":434,"threads_started":1}
I20260811 18:40:31.151273 2922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:34685 every 8 connection(s)
I20260811 18:40:31.151294 2666 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:34685
I20260811 18:40:31.151922 2462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.130579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:39851 (local address 127.30.185.190:42283)
0811 18:40:31.130772 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:31.130777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:31.131578 (+ 801us) server_negotiation.cc:407] Connection header received
0811 18:40:31.131810 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:31.131815 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:31.131936 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:31.132061 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:31.134056 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.135043 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.138324 (+ 3281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.138996 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.139158 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.142730 (+ 3572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:31.142734 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:31.142738 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:31.142771 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:31.147554 (+ 4783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:31.148412 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:31.148418 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:31.148422 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:31.148582 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:31.149036 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:31.149040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:31.149046 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:31.149396 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:31.149553 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:31.151278 (+ 1725us) server_negotiation.cc:299] Negotiation successful
0811 18:40:31.151582 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 18:40:31.153249 2666 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:40:31.156909 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 2666
I20260811 18:40:31.156137 2923 heartbeater.cc:347] Connected to a master server at 127.30.185.190:42283
I20260811 18:40:31.157197 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:40:31.157411 2923 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:31.157709 31462 external_mini_cluster.cc:1428] Setting key 0bb0bf6dac75576ad2c5d81c3f82753a
I20260811 18:40:31.158293 2923 heartbeater.cc:511] Master 127.30.185.190:42283 requested a full tablet report, sending...
I20260811 18:40:31.160209 2042 ts_manager.cc:194] Registered new tserver with Master: e4133e575b754c4fb437792fca3afd09 (127.30.185.131:34685)
I20260811 18:40:31.161268 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:40:31.161454 2042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:35835
Aug 11 18:40:31 dist-test-slave-z8x9 krb5kdc[1968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:40:31 dist-test-slave-z8x9 krb5kdc[1968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:31.224257 2462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.204426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55564 (local address 127.30.185.190:42283)
0811 18:40:31.204727 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:31.204734 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:31.205020 (+ 286us) server_negotiation.cc:407] Connection header received
0811 18:40:31.205211 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:31.205219 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:31.205292 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:31.205386 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:31.207034 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.207781 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.209343 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.209740 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.209844 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.214183 (+ 4339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:31.214188 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:31.214191 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:31.214227 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:31.220994 (+ 6767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:31.221859 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:31.221863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:31.221866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:31.222097 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:31.222521 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:31.222524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:31.222526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:31.222939 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:31.223318 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:31.223600 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 18:40:31.223849 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260811 18:40:31.229359 2042 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:40:31.237433 2042 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:40:31.278551 2957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.252399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:45641 (local address 127.0.0.1:34644)
0811 18:40:31.267422 (+ 15023us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:31.267445 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:31.267620 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:31.268653 (+ 1033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:31.268664 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:31.268715 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:31.269705 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.269730 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.271841 (+ 2111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.271850 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.275065 (+ 3215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.275084 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.277263 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.277273 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.277555 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.277595 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 18:40:31.277737 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 18:40:31.277946 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14351,"mutex_wait_us":51,"thread_start_us":17657,"threads_started":1}
I20260811 18:40:31.278657 2952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.251829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:34685 (local address 127.0.0.1:56058)
0811 18:40:31.252994 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:31.253038 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:31.253419 (+ 381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:31.269221 (+ 15802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:31.269229 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:31.269260 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:31.270221 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.270246 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.272784 (+ 2538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.272790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.275599 (+ 2809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.275621 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.277828 (+ 2207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.277834 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.278002 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.278032 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 18:40:31.278125 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 18:40:31.278258 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":931,"mutex_wait_us":26,"thread_start_us":789,"threads_started":1}
I20260811 18:40:31.279363 2956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.253216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34644 (local address 127.30.185.129:45641)
0811 18:40:31.264141 (+ 10925us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:31.264155 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:31.267834 (+ 3679us) server_negotiation.cc:407] Connection header received
0811 18:40:31.267964 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:31.267974 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:31.268344 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:31.268582 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:31.270214 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.271676 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.275297 (+ 3621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.276863 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.277088 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.277368 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:31.278851 (+ 1483us) server_negotiation.cc:299] Negotiation successful
0811 18:40:31.279082 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10684,"thread_start_us":10453,"threads_started":1}
I20260811 18:40:31.279876 2955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.252418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56058 (local address 127.30.185.131:34685)
0811 18:40:31.268636 (+ 16218us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:31.268645 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:31.268694 (+ 49us) server_negotiation.cc:407] Connection header received
0811 18:40:31.268802 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:31.268810 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:31.268993 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:31.269255 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:31.270547 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.272616 (+ 2069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.275788 (+ 3172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.277592 (+ 1804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.277793 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.277950 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:31.278228 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 18:40:31.279488 (+ 1260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16033,"thread_start_us":15793,"threads_started":1}
I20260811 18:40:31.280689 2959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.254011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56268 (local address 127.30.185.130:44815)
0811 18:40:31.267637 (+ 13626us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:31.267646 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:31.270654 (+ 3008us) server_negotiation.cc:407] Connection header received
0811 18:40:31.271056 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:31.271066 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:31.271400 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:31.271577 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:31.272923 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.274105 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.277325 (+ 3220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.279084 (+ 1759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.279227 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.279404 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:31.280268 (+ 864us) server_negotiation.cc:299] Negotiation successful
0811 18:40:31.280424 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13438,"thread_start_us":13631,"threads_started":1}
I20260811 18:40:31.280844 2954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.253493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:44815 (local address 127.0.0.1:56268)
0811 18:40:31.270532 (+ 17039us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:31.270551 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:31.270770 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:31.271804 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:31.271809 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:31.271833 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:31.272543 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.272558 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.274324 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.274330 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.277086 (+ 2756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.277101 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.279863 (+ 2762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.279871 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.280054 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.280096 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 18:40:31.280206 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 18:40:31.280337 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16895,"mutex_wait_us":31,"thread_start_us":13495,"threads_started":1}
I20260811 18:40:31.283582 2794 tablet_service.cc:1467] Processing CreateTablet for tablet 977b373d3d134811a0ac51b31ca0c377 (DEFAULT_TABLE table=test-table [id=48dd3bd7e805475bb64aaa811e528a02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:31.283578 2291 tablet_service.cc:1467] Processing CreateTablet for tablet 977b373d3d134811a0ac51b31ca0c377 (DEFAULT_TABLE table=test-table [id=48dd3bd7e805475bb64aaa811e528a02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:31.284245 2543 tablet_service.cc:1467] Processing CreateTablet for tablet 977b373d3d134811a0ac51b31ca0c377 (DEFAULT_TABLE table=test-table [id=48dd3bd7e805475bb64aaa811e528a02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:31.284946 2794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 977b373d3d134811a0ac51b31ca0c377. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:31.285060 2291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 977b373d3d134811a0ac51b31ca0c377. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:31.285758 2543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 977b373d3d134811a0ac51b31ca0c377. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:31.299589 2961 tablet_bootstrap.cc:492] T 977b373d3d134811a0ac51b31ca0c377 P e4133e575b754c4fb437792fca3afd09: Bootstrap starting.
I20260811 18:40:31.301980 2964 tablet_bootstrap.cc:492] T 977b373d3d134811a0ac51b31ca0c377 P 7209845a058349c6a108c7fa04c5b580: Bootstrap starting.
I20260811 18:40:31.304332 2961 tablet_bootstrap.cc:654] T 977b373d3d134811a0ac51b31ca0c377 P e4133e575b754c4fb437792fca3afd09: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:31.305902 2966 tablet_bootstrap.cc:492] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7: Bootstrap starting.
I20260811 18:40:31.306010 2961 log.cc:824] T 977b373d3d134811a0ac51b31ca0c377 P e4133e575b754c4fb437792fca3afd09: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:31.306396 2964 tablet_bootstrap.cc:654] T 977b373d3d134811a0ac51b31ca0c377 P 7209845a058349c6a108c7fa04c5b580: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:31.308367 2964 log.cc:824] T 977b373d3d134811a0ac51b31ca0c377 P 7209845a058349c6a108c7fa04c5b580: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:31.310408 2966 tablet_bootstrap.cc:654] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:31.310947 2961 tablet_bootstrap.cc:492] T 977b373d3d134811a0ac51b31ca0c377 P e4133e575b754c4fb437792fca3afd09: No bootstrap required, opened a new log
I20260811 18:40:31.311242 2961 ts_tablet_manager.cc:1397] T 977b373d3d134811a0ac51b31ca0c377 P e4133e575b754c4fb437792fca3afd09: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260811 18:40:31.311504 2966 log.cc:824] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:31.311909 2964 tablet_bootstrap.cc:492] T 977b373d3d134811a0ac51b31ca0c377 P 7209845a058349c6a108c7fa04c5b580: No bootstrap required, opened a new log
I20260811 18:40:31.312196 2964 ts_tablet_manager.cc:1397] T 977b373d3d134811a0ac51b31ca0c377 P 7209845a058349c6a108c7fa04c5b580: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260811 18:40:31.314342 2966 tablet_bootstrap.cc:492] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7: No bootstrap required, opened a new log
I20260811 18:40:31.314591 2966 ts_tablet_manager.cc:1397] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 18:40:31.323447 2966 raft_consensus.cc:348] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } } peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } }
I20260811 18:40:31.323825 2966 raft_consensus.cc:374] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:31.323958 2966 raft_consensus.cc:729] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1e13f628490435d85fa802f60e2d0f7, State: Initialized, Role: FOLLOWER
I20260811 18:40:31.323927 2961 raft_consensus.cc:348] T 977b373d3d134811a0ac51b31ca0c377 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } } peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } }
I20260811 18:40:31.324373 2961 raft_consensus.cc:374] T 977b373d3d134811a0ac51b31ca0c377 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:31.324527 2961 raft_consensus.cc:729] T 977b373d3d134811a0ac51b31ca0c377 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4133e575b754c4fb437792fca3afd09, State: Initialized, Role: FOLLOWER
I20260811 18:40:31.324608 2966 consensus_queue.cc:260] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } } peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } }
I20260811 18:40:31.324551 2964 raft_consensus.cc:348] T 977b373d3d134811a0ac51b31ca0c377 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } } peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } }
I20260811 18:40:31.324983 2964 raft_consensus.cc:374] T 977b373d3d134811a0ac51b31ca0c377 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:31.325102 2964 raft_consensus.cc:729] T 977b373d3d134811a0ac51b31ca0c377 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7209845a058349c6a108c7fa04c5b580, State: Initialized, Role: FOLLOWER
I20260811 18:40:31.325276 2961 consensus_queue.cc:260] T 977b373d3d134811a0ac51b31ca0c377 P e4133e575b754c4fb437792fca3afd09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } } peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } }
I20260811 18:40:31.326028 2964 consensus_queue.cc:260] T 977b373d3d134811a0ac51b31ca0c377 P 7209845a058349c6a108c7fa04c5b580 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } } peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } }
I20260811 18:40:31.328478 2964 ts_tablet_manager.cc:1428] T 977b373d3d134811a0ac51b31ca0c377 P 7209845a058349c6a108c7fa04c5b580: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260811 18:40:31.329507 2923 heartbeater.cc:503] Master 127.30.185.190:42283 was elected leader, sending a full tablet report...
I20260811 18:40:31.330119 2961 ts_tablet_manager.cc:1428] T 977b373d3d134811a0ac51b31ca0c377 P e4133e575b754c4fb437792fca3afd09: Time spent starting tablet: real 0.019s user 0.015s sys 0.005s
I20260811 18:40:31.331566 2662 heartbeater.cc:503] Master 127.30.185.190:42283 was elected leader, sending a full tablet report...
I20260811 18:40:31.332118 2966 ts_tablet_manager.cc:1428] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7: Time spent starting tablet: real 0.017s user 0.008s sys 0.008s
W20260811 18:40:31.355656 2663 tablet.cc:2369] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:40:31.407167 2925 tablet.cc:2369] T 977b373d3d134811a0ac51b31ca0c377 P e4133e575b754c4fb437792fca3afd09: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:40:31.476207 2973 raft_consensus.cc:482] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:40:31.476680 2973 raft_consensus.cc:504] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } } peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } }
I20260811 18:40:31.478552 2973 leader_election.cc:288] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4133e575b754c4fb437792fca3afd09 (127.30.185.131:34685), 7209845a058349c6a108c7fa04c5b580 (127.30.185.129:45641)
I20260811 18:40:31.490160 2956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.479090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:42881 (local address 127.30.185.129:45641)
0811 18:40:31.479451 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:31.479468 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:31.480642 (+ 1174us) server_negotiation.cc:407] Connection header received
0811 18:40:31.480909 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:31.480913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:31.481016 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:31.481133 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:31.482313 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.483838 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.487207 (+ 3369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.488866 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.489068 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.489145 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:31.489710 (+ 565us) server_negotiation.cc:299] Negotiation successful
0811 18:40:31.489845 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260811 18:40:31.490283 2955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.479502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:48937 (local address 127.30.185.131:34685)
0811 18:40:31.479826 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:31.479834 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:31.480851 (+ 1017us) server_negotiation.cc:407] Connection header received
0811 18:40:31.481017 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:31.481023 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:31.481107 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:31.481229 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:31.482541 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.483883 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.487219 (+ 3336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.488866 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.489079 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.489145 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:31.489604 (+ 459us) server_negotiation.cc:299] Negotiation successful
0811 18:40:31.489738 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260811 18:40:31.490450 2979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.479135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:34685 (local address 127.30.185.130:48937)
0811 18:40:31.480712 (+ 1577us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:31.480729 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:31.480887 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:31.481505 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:31.481511 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:31.481569 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:31.482354 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.482364 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.484002 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.484006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.486977 (+ 2971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.486993 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.489161 (+ 2168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.489195 (+ 34us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.489400 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.489439 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 18:40:31.489559 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 18:40:31.489722 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1402,"mutex_wait_us":206,"thread_start_us":689,"threads_started":1}
I20260811 18:40:31.490748 2980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.478994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:45641 (local address 127.30.185.130:42881)
0811 18:40:31.480490 (+ 1496us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:31.480524 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:31.480721 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:31.481145 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:31.481148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:31.481215 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:31.482109 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.482127 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.483998 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.484003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.486977 (+ 2974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.486993 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.489407 (+ 2414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.489413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.489531 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.489555 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:40:31.489682 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 18:40:31.489816 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1232,"mutex_wait_us":207,"thread_start_us":1234,"threads_started":1}
I20260811 18:40:31.491791 2833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977b373d3d134811a0ac51b31ca0c377" candidate_uuid: "a1e13f628490435d85fa802f60e2d0f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4133e575b754c4fb437792fca3afd09" is_pre_election: true
I20260811 18:40:31.491865 2324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977b373d3d134811a0ac51b31ca0c377" candidate_uuid: "a1e13f628490435d85fa802f60e2d0f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7209845a058349c6a108c7fa04c5b580" is_pre_election: true
I20260811 18:40:31.492750 2324 raft_consensus.cc:2454] T 977b373d3d134811a0ac51b31ca0c377 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1e13f628490435d85fa802f60e2d0f7 in term 0.
I20260811 18:40:31.492754 2833 raft_consensus.cc:2454] T 977b373d3d134811a0ac51b31ca0c377 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1e13f628490435d85fa802f60e2d0f7 in term 0.
I20260811 18:40:31.493449 2450 leader_election.cc:302] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1e13f628490435d85fa802f60e2d0f7, e4133e575b754c4fb437792fca3afd09; no voters:
I20260811 18:40:31.493808 2973 raft_consensus.cc:2781] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:40:31.494009 2973 raft_consensus.cc:482] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:40:31.494140 2973 raft_consensus.cc:3037] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:31.496825 2973 raft_consensus.cc:504] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } } peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } }
I20260811 18:40:31.497545 2973 leader_election.cc:288] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7 [CANDIDATE]: Term 1 election: Requested vote from peers e4133e575b754c4fb437792fca3afd09 (127.30.185.131:34685), 7209845a058349c6a108c7fa04c5b580 (127.30.185.129:45641)
I20260811 18:40:31.498142 2833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977b373d3d134811a0ac51b31ca0c377" candidate_uuid: "a1e13f628490435d85fa802f60e2d0f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4133e575b754c4fb437792fca3afd09"
I20260811 18:40:31.498142 2324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977b373d3d134811a0ac51b31ca0c377" candidate_uuid: "a1e13f628490435d85fa802f60e2d0f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7209845a058349c6a108c7fa04c5b580"
I20260811 18:40:31.498371 2833 raft_consensus.cc:3037] T 977b373d3d134811a0ac51b31ca0c377 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:31.498400 2324 raft_consensus.cc:3037] T 977b373d3d134811a0ac51b31ca0c377 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:31.502055 2324 raft_consensus.cc:2454] T 977b373d3d134811a0ac51b31ca0c377 P 7209845a058349c6a108c7fa04c5b580 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1e13f628490435d85fa802f60e2d0f7 in term 1.
I20260811 18:40:31.502071 2833 raft_consensus.cc:2454] T 977b373d3d134811a0ac51b31ca0c377 P e4133e575b754c4fb437792fca3afd09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1e13f628490435d85fa802f60e2d0f7 in term 1.
I20260811 18:40:31.502712 2452 leader_election.cc:302] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7209845a058349c6a108c7fa04c5b580, a1e13f628490435d85fa802f60e2d0f7; no voters:
I20260811 18:40:31.503151 2973 raft_consensus.cc:2781] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:40:31.503750 2973 raft_consensus.cc:686] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7 [term 1 LEADER]: Becoming Leader. State: Replica: a1e13f628490435d85fa802f60e2d0f7, State: Running, Role: LEADER
I20260811 18:40:31.504247 2973 consensus_queue.cc:237] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } } peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } }
I20260811 18:40:31.509454 2040 catalog_manager.cc:5495] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1e13f628490435d85fa802f60e2d0f7 (127.30.185.130). New cstate: current_term: 1 leader_uuid: "a1e13f628490435d85fa802f60e2d0f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } health_report { overall_health: HEALTHY } } }
W20260811 18:40:31.543118 2416 tablet.cc:2369] T 977b373d3d134811a0ac51b31ca0c377 P 7209845a058349c6a108c7fa04c5b580: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 18:40:31 dist-test-slave-z8x9 krb5kdc[1968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473629, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:31.612709 2462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.596139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55566 (local address 127.30.185.190:42283)
0811 18:40:31.596417 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:31.596422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:31.597287 (+ 865us) server_negotiation.cc:407] Connection header received
0811 18:40:31.597507 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:31.597512 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:31.597590 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:31.597697 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:31.599174 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.599977 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.601606 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.602081 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.602229 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.606270 (+ 4041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:31.606275 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:31.606277 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:31.606308 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:31.609366 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:31.610213 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:31.610225 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:31.610227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:31.610407 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:31.610802 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:31.610806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:31.610808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:31.611108 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:31.611301 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:31.611820 (+ 519us) server_negotiation.cc:299] Negotiation successful
0811 18:40:31.612164 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":119,"server-negotiator.queue_time_us":132}
I20260811 18:40:31.612783 2995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.596406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:42283 (local address 127.0.0.1:55566)
0811 18:40:31.597117 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:31.597147 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:31.597375 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:31.597723 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:31.597727 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:31.598578 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:31.599018 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.599029 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.600153 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.600157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.601469 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.601482 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.602390 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.602397 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.602560 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.603289 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:31.603306 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:31.606079 (+ 2773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:31.609501 (+ 3422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:31.609514 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:31.609522 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:31.610069 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:31.610523 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:31.610529 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:31.610534 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:31.610675 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:31.611259 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:31.611273 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:31.611487 (+ 214us) client_negotiation.cc:769] Sending connection context
0811 18:40:31.611699 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 18:40:31.612018 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":374,"threads_started":1}
I20260811 18:40:31.618940 2040 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:55566:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 18:40:31.620282 2040 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:40:31.628021 2291 tablet_service.cc:1467] Processing CreateTablet for tablet f3906c487520482fafc37d3bfdfc1f48 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0cd4da5db6cb4f76b2915faa670dcd74]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:40:31.628276 2543 tablet_service.cc:1467] Processing CreateTablet for tablet f3906c487520482fafc37d3bfdfc1f48 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0cd4da5db6cb4f76b2915faa670dcd74]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:40:31.628557 2794 tablet_service.cc:1467] Processing CreateTablet for tablet f3906c487520482fafc37d3bfdfc1f48 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0cd4da5db6cb4f76b2915faa670dcd74]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:40:31.629287 2291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3906c487520482fafc37d3bfdfc1f48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:31.629364 2543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3906c487520482fafc37d3bfdfc1f48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:31.629436 2794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3906c487520482fafc37d3bfdfc1f48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:31.635725 2961 tablet_bootstrap.cc:492] T f3906c487520482fafc37d3bfdfc1f48 P e4133e575b754c4fb437792fca3afd09: Bootstrap starting.
I20260811 18:40:31.637030 2966 tablet_bootstrap.cc:492] T f3906c487520482fafc37d3bfdfc1f48 P a1e13f628490435d85fa802f60e2d0f7: Bootstrap starting.
I20260811 18:40:31.637880 2961 tablet_bootstrap.cc:654] T f3906c487520482fafc37d3bfdfc1f48 P e4133e575b754c4fb437792fca3afd09: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:31.638652 2966 tablet_bootstrap.cc:654] T f3906c487520482fafc37d3bfdfc1f48 P a1e13f628490435d85fa802f60e2d0f7: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:31.640621 2964 tablet_bootstrap.cc:492] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580: Bootstrap starting.
I20260811 18:40:31.640755 2961 tablet_bootstrap.cc:492] T f3906c487520482fafc37d3bfdfc1f48 P e4133e575b754c4fb437792fca3afd09: No bootstrap required, opened a new log
I20260811 18:40:31.640908 2961 ts_tablet_manager.cc:1397] T f3906c487520482fafc37d3bfdfc1f48 P e4133e575b754c4fb437792fca3afd09: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260811 18:40:31.641912 2966 tablet_bootstrap.cc:492] T f3906c487520482fafc37d3bfdfc1f48 P a1e13f628490435d85fa802f60e2d0f7: No bootstrap required, opened a new log
I20260811 18:40:31.642086 2966 ts_tablet_manager.cc:1397] T f3906c487520482fafc37d3bfdfc1f48 P a1e13f628490435d85fa802f60e2d0f7: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 18:40:31.642515 2964 tablet_bootstrap.cc:654] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:31.642191 2961 raft_consensus.cc:348] T f3906c487520482fafc37d3bfdfc1f48 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:31.642658 2961 raft_consensus.cc:374] T f3906c487520482fafc37d3bfdfc1f48 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:31.642800 2961 raft_consensus.cc:729] T f3906c487520482fafc37d3bfdfc1f48 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4133e575b754c4fb437792fca3afd09, State: Initialized, Role: FOLLOWER
I20260811 18:40:31.643069 2961 consensus_queue.cc:260] T f3906c487520482fafc37d3bfdfc1f48 P e4133e575b754c4fb437792fca3afd09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:31.643297 2966 raft_consensus.cc:348] T f3906c487520482fafc37d3bfdfc1f48 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:31.643558 2966 raft_consensus.cc:374] T f3906c487520482fafc37d3bfdfc1f48 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:31.643682 2966 raft_consensus.cc:729] T f3906c487520482fafc37d3bfdfc1f48 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1e13f628490435d85fa802f60e2d0f7, State: Initialized, Role: FOLLOWER
I20260811 18:40:31.643781 2975 tablet_replica.cc:439] T f3906c487520482fafc37d3bfdfc1f48 P e4133e575b754c4fb437792fca3afd09: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } } }
I20260811 18:40:31.643932 2966 consensus_queue.cc:260] T f3906c487520482fafc37d3bfdfc1f48 P a1e13f628490435d85fa802f60e2d0f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:31.644116 2975 tablet_replica.cc:442] T f3906c487520482fafc37d3bfdfc1f48 P e4133e575b754c4fb437792fca3afd09: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:40:31.644438 2973 tablet_replica.cc:439] T f3906c487520482fafc37d3bfdfc1f48 P a1e13f628490435d85fa802f60e2d0f7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } } }
I20260811 18:40:31.644832 2961 ts_tablet_manager.cc:1428] T f3906c487520482fafc37d3bfdfc1f48 P e4133e575b754c4fb437792fca3afd09: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260811 18:40:31.646097 2966 ts_tablet_manager.cc:1428] T f3906c487520482fafc37d3bfdfc1f48 P a1e13f628490435d85fa802f60e2d0f7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:40:31.646353 2973 tablet_replica.cc:442] T f3906c487520482fafc37d3bfdfc1f48 P a1e13f628490435d85fa802f60e2d0f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:40:31.647356 2964 tablet_bootstrap.cc:492] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580: No bootstrap required, opened a new log
I20260811 18:40:31.647524 2964 ts_tablet_manager.cc:1397] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260811 18:40:31.648730 2964 raft_consensus.cc:348] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:31.649044 2964 raft_consensus.cc:374] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:31.649158 2964 raft_consensus.cc:729] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7209845a058349c6a108c7fa04c5b580, State: Initialized, Role: FOLLOWER
I20260811 18:40:31.649463 2964 consensus_queue.cc:260] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:31.650136 2964 ts_tablet_manager.cc:1428] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 18:40:31.649956 2977 tablet_replica.cc:439] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } } }
I20260811 18:40:31.650400 2977 tablet_replica.cc:442] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:40:31.702018 2977 raft_consensus.cc:482] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:40:31.702363 2977 raft_consensus.cc:504] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:31.704464 2977 leader_election.cc:288] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1e13f628490435d85fa802f60e2d0f7 (127.30.185.130:44815), e4133e575b754c4fb437792fca3afd09 (127.30.185.131:34685)
I20260811 18:40:31.714705 2955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.706105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:35681 (local address 127.30.185.131:34685)
0811 18:40:31.706335 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:31.706342 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:31.706805 (+ 463us) server_negotiation.cc:407] Connection header received
0811 18:40:31.707007 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:31.707011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:31.707075 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:31.707146 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:31.707960 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.708944 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.711998 (+ 3054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.713468 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.713631 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.713710 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:31.714309 (+ 599us) server_negotiation.cc:299] Negotiation successful
0811 18:40:31.714435 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 18:40:31.714748 2959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.705252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:51273 (local address 127.30.185.130:44815)
0811 18:40:31.705531 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:31.705540 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:31.705884 (+ 344us) server_negotiation.cc:407] Connection header received
0811 18:40:31.706174 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:31.706194 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:31.706299 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:31.706411 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:31.707308 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.708515 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.711712 (+ 3197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.713460 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.713637 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.713710 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:31.714182 (+ 472us) server_negotiation.cc:299] Negotiation successful
0811 18:40:31.714343 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 18:40:31.715060 3000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.705026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:44815 (local address 127.30.185.129:51273)
0811 18:40:31.705768 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:31.705785 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:31.705985 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:31.706441 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:31.706445 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:31.706474 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:31.707071 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.707082 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.708651 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.708654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.711475 (+ 2821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.711494 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.713721 (+ 2227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.713729 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.713982 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.714027 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 18:40:31.714132 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 18:40:31.714291 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":28,"thread_start_us":452,"threads_started":1}
I20260811 18:40:31.715626 3001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.705943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:34685 (local address 127.30.185.129:35681)
0811 18:40:31.706662 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:31.706693 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:31.706842 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:31.707192 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:31.707196 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:31.707223 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:31.707708 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.707719 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.709086 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.709091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.711780 (+ 2689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.711792 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.713932 (+ 2140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.713940 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.714115 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.714149 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 18:40:31.714227 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 18:40:31.714326 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":4,"thread_start_us":293,"threads_started":1}
I20260811 18:40:31.716398 2833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3906c487520482fafc37d3bfdfc1f48" candidate_uuid: "7209845a058349c6a108c7fa04c5b580" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4133e575b754c4fb437792fca3afd09" is_pre_election: true
I20260811 18:40:31.716228 2587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3906c487520482fafc37d3bfdfc1f48" candidate_uuid: "7209845a058349c6a108c7fa04c5b580" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1e13f628490435d85fa802f60e2d0f7" is_pre_election: true
I20260811 18:40:31.716725 2833 raft_consensus.cc:2454] T f3906c487520482fafc37d3bfdfc1f48 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7209845a058349c6a108c7fa04c5b580 in term 0.
I20260811 18:40:31.716895 2587 raft_consensus.cc:2454] T f3906c487520482fafc37d3bfdfc1f48 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7209845a058349c6a108c7fa04c5b580 in term 0.
I20260811 18:40:31.717425 2197 leader_election.cc:302] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7209845a058349c6a108c7fa04c5b580, e4133e575b754c4fb437792fca3afd09; no voters:
I20260811 18:40:31.717933 2977 raft_consensus.cc:2781] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:40:31.718164 2977 raft_consensus.cc:482] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:40:31.718398 2977 raft_consensus.cc:3037] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:31.721378 2977 raft_consensus.cc:504] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:31.721982 2977 leader_election.cc:288] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580 [CANDIDATE]: Term 1 election: Requested vote from peers a1e13f628490435d85fa802f60e2d0f7 (127.30.185.130:44815), e4133e575b754c4fb437792fca3afd09 (127.30.185.131:34685)
I20260811 18:40:31.722620 2587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3906c487520482fafc37d3bfdfc1f48" candidate_uuid: "7209845a058349c6a108c7fa04c5b580" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1e13f628490435d85fa802f60e2d0f7"
I20260811 18:40:31.722730 2833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3906c487520482fafc37d3bfdfc1f48" candidate_uuid: "7209845a058349c6a108c7fa04c5b580" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4133e575b754c4fb437792fca3afd09"
I20260811 18:40:31.722856 2587 raft_consensus.cc:3037] T f3906c487520482fafc37d3bfdfc1f48 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:31.723060 2833 raft_consensus.cc:3037] T f3906c487520482fafc37d3bfdfc1f48 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:31.725787 2833 raft_consensus.cc:2454] T f3906c487520482fafc37d3bfdfc1f48 P e4133e575b754c4fb437792fca3afd09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7209845a058349c6a108c7fa04c5b580 in term 1.
I20260811 18:40:31.726348 2197 leader_election.cc:302] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7209845a058349c6a108c7fa04c5b580, e4133e575b754c4fb437792fca3afd09; no voters:
I20260811 18:40:31.726672 2977 raft_consensus.cc:2781] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:40:31.726861 2587 raft_consensus.cc:2454] T f3906c487520482fafc37d3bfdfc1f48 P a1e13f628490435d85fa802f60e2d0f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7209845a058349c6a108c7fa04c5b580 in term 1.
I20260811 18:40:31.727279 2977 raft_consensus.cc:686] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580 [term 1 LEADER]: Becoming Leader. State: Replica: 7209845a058349c6a108c7fa04c5b580, State: Running, Role: LEADER
I20260811 18:40:31.727860 2977 consensus_queue.cc:237] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:31.731451 3002 tablet_replica.cc:439] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580: TxnStatusTablet state changed. Reason: New leader 7209845a058349c6a108c7fa04c5b580. Latest consensus state: current_term: 1 leader_uuid: "7209845a058349c6a108c7fa04c5b580" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } } }
I20260811 18:40:31.732103 3002 tablet_replica.cc:442] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:40:31.733142 3006 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 18:40:31.733532 3006 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 18:40:31.732825 2043 catalog_manager.cc:5495] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7209845a058349c6a108c7fa04c5b580 (127.30.185.129). New cstate: current_term: 1 leader_uuid: "7209845a058349c6a108c7fa04c5b580" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } health_report { overall_health: UNKNOWN } } }
I20260811 18:40:31.756608 2995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.749685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:45641 (local address 127.0.0.1:34646)
0811 18:40:31.750070 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:31.750102 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:31.750346 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:31.751014 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:31.751018 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:31.751028 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:40:31.751189 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.751196 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.752675 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.752679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.753795 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.753809 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.754783 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.754787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.755044 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.755104 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:40:31.756167 (+ 1063us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:40:31.756185 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 18:40:31.756239 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 18:40:31.756368 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 18:40:31.756728 2956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.750391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34646 (local address 127.30.185.129:45641)
0811 18:40:31.750605 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:31.750612 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:31.750633 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:40:31.750726 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:31.750731 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:31.750840 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:31.750997 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:40:31.751326 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.752528 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.753942 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.754598 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.754748 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.755318 (+ 570us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:40:31.756057 (+ 739us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:40:31.756212 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:31.756276 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 18:40:31.756364 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 18:40:31.765590 2587 raft_consensus.cc:1261] T f3906c487520482fafc37d3bfdfc1f48 P a1e13f628490435d85fa802f60e2d0f7 [term 1 FOLLOWER]: Refusing update from remote peer 7209845a058349c6a108c7fa04c5b580: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:40:31.765676 2833 raft_consensus.cc:1261] T f3906c487520482fafc37d3bfdfc1f48 P e4133e575b754c4fb437792fca3afd09 [term 1 FOLLOWER]: Refusing update from remote peer 7209845a058349c6a108c7fa04c5b580: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:40:31.766850 3002 consensus_queue.cc:1035] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:40:31.767314 2977 consensus_queue.cc:1035] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:40:31.781245 2975 tablet_replica.cc:439] T f3906c487520482fafc37d3bfdfc1f48 P e4133e575b754c4fb437792fca3afd09: TxnStatusTablet state changed. Reason: New leader 7209845a058349c6a108c7fa04c5b580. Latest consensus state: current_term: 1 leader_uuid: "7209845a058349c6a108c7fa04c5b580" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } } }
I20260811 18:40:31.781553 2975 tablet_replica.cc:442] T f3906c487520482fafc37d3bfdfc1f48 P e4133e575b754c4fb437792fca3afd09: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:40:31.784094 3009 mvcc.cc:205] Tried to move back new op lower bound from 7317395995699793920 to 7317395995561381888. Current Snapshot: MvccSnapshot[applied={T|T < 7317395995699793920}]
I20260811 18:40:31.784919 3002 tablet_replica.cc:439] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7209845a058349c6a108c7fa04c5b580" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } } }
I20260811 18:40:31.785287 3002 tablet_replica.cc:442] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:40:31.790066 2975 tablet_replica.cc:439] T f3906c487520482fafc37d3bfdfc1f48 P e4133e575b754c4fb437792fca3afd09: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7209845a058349c6a108c7fa04c5b580" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } } }
I20260811 18:40:31.790391 2975 tablet_replica.cc:442] T f3906c487520482fafc37d3bfdfc1f48 P e4133e575b754c4fb437792fca3afd09: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:40:31.792711 2973 tablet_replica.cc:439] T f3906c487520482fafc37d3bfdfc1f48 P a1e13f628490435d85fa802f60e2d0f7: TxnStatusTablet state changed. Reason: New leader 7209845a058349c6a108c7fa04c5b580. Latest consensus state: current_term: 1 leader_uuid: "7209845a058349c6a108c7fa04c5b580" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } } }
I20260811 18:40:31.793761 2973 tablet_replica.cc:442] T f3906c487520482fafc37d3bfdfc1f48 P a1e13f628490435d85fa802f60e2d0f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:40:31.793864 3002 tablet_replica.cc:439] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7209845a058349c6a108c7fa04c5b580" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } } }
I20260811 18:40:31.794198 3002 tablet_replica.cc:442] T f3906c487520482fafc37d3bfdfc1f48 P 7209845a058349c6a108c7fa04c5b580: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:40:31.806470 2973 tablet_replica.cc:439] T f3906c487520482fafc37d3bfdfc1f48 P a1e13f628490435d85fa802f60e2d0f7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7209845a058349c6a108c7fa04c5b580" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } } }
I20260811 18:40:31.806834 2973 tablet_replica.cc:442] T f3906c487520482fafc37d3bfdfc1f48 P a1e13f628490435d85fa802f60e2d0f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:40:31.816581 2959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.806803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56272 (local address 127.30.185.130:44815)
0811 18:40:31.807030 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:31.807040 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:31.810374 (+ 3334us) server_negotiation.cc:407] Connection header received
0811 18:40:31.810730 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:31.810735 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:31.810813 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:31.810977 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:40:31.811513 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.812339 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.814278 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.814660 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.814804 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.815277 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:40:31.815835 (+ 558us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:40:31.815995 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:31.816096 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 18:40:31.816190 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 18:40:31.837359 3031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.829745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:45641 (local address 127.30.185.130:44895)
0811 18:40:31.830612 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:31.830629 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:31.830757 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:31.831163 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:31.831168 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:31.831183 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:40:31.831522 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.831532 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.833341 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.833345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.834555 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.834567 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.835931 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.835935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.836095 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.836158 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:40:31.836697 (+ 539us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:40:31.836715 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 18:40:31.836780 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 18:40:31.836927 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"thread_start_us":561,"threads_started":1}
I20260811 18:40:31.837805 2956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.829610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:44895 (local address 127.30.185.129:45641)
0811 18:40:31.829799 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:31.829805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:31.830730 (+ 925us) server_negotiation.cc:407] Connection header received
0811 18:40:31.830900 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:31.830906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:31.831017 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:31.831117 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:40:31.831683 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.833115 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.834793 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.835668 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.835892 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.836401 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:40:31.836570 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:40:31.836705 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:31.836841 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 18:40:31.836969 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 18:40:31.848120 2324 raft_consensus.cc:1261] T 977b373d3d134811a0ac51b31ca0c377 P 7209845a058349c6a108c7fa04c5b580 [term 1 FOLLOWER]: Refusing update from remote peer a1e13f628490435d85fa802f60e2d0f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:40:31.849124 2833 raft_consensus.cc:1261] T 977b373d3d134811a0ac51b31ca0c377 P e4133e575b754c4fb437792fca3afd09 [term 1 FOLLOWER]: Refusing update from remote peer a1e13f628490435d85fa802f60e2d0f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:40:31.849159 2981 consensus_queue.cc:1035] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 18:40:31.849917 2973 consensus_queue.cc:1035] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:40:31.857076 3010 mvcc.cc:205] Tried to move back new op lower bound from 7317395996041748480 to 7317395994645245952. Current Snapshot: MvccSnapshot[applied={T|T < 7317395996041748480}]
I20260811 18:40:31.859543 3011 mvcc.cc:205] Tried to move back new op lower bound from 7317395996041748480 to 7317395994645245952. Current Snapshot: MvccSnapshot[applied={T|T < 7317395996041748480}]
I20260811 18:40:31.895263 2959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.884197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:41375 (local address 127.30.185.130:44815)
0811 18:40:31.884425 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:31.884431 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:31.887494 (+ 3063us) server_negotiation.cc:407] Connection header received
0811 18:40:31.887811 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:31.887817 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:31.887905 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:31.887980 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:40:31.889945 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.890986 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.892874 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.893700 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.893851 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.894413 (+ 562us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:40:31.894602 (+ 189us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:40:31.894721 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:31.894883 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 18:40:31.894972 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 18:40:31.895825 3039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:31.884123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:44815 (local address 127.30.185.129:41375)
0811 18:40:31.887394 (+ 3271us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:31.887416 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:31.887617 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:31.888905 (+ 1288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:31.888909 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:31.888922 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:40:31.889731 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.889747 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.891135 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.891140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.892710 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:31.892724 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:31.893920 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:31.893928 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:31.894117 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:31.894196 (+ 79us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:40:31.894728 (+ 532us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:40:31.894740 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 18:40:31.894835 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 18:40:31.894985 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3041,"thread_start_us":1104,"threads_started":1}
I20260811 18:40:31.942094 2040 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:40:31.942531 2040 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:40:31.947315 2040 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435: Sending DeleteTablet for 3 replicas of tablet 977b373d3d134811a0ac51b31ca0c377
I20260811 18:40:31.948920 2794 tablet_service.cc:1514] Processing DeleteTablet for tablet 977b373d3d134811a0ac51b31ca0c377 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:31 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:56058
I20260811 18:40:31.949117 2543 tablet_service.cc:1514] Processing DeleteTablet for tablet 977b373d3d134811a0ac51b31ca0c377 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:31 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:56268
I20260811 18:40:31.950160 3045 tablet_replica.cc:330] T 977b373d3d134811a0ac51b31ca0c377 P e4133e575b754c4fb437792fca3afd09: stopping tablet replica
I20260811 18:40:31.950582 3045 raft_consensus.cc:2229] T 977b373d3d134811a0ac51b31ca0c377 P e4133e575b754c4fb437792fca3afd09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:40:31.950815 2291 tablet_service.cc:1514] Processing DeleteTablet for tablet 977b373d3d134811a0ac51b31ca0c377 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:31 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:34644
I20260811 18:40:31.951051 3045 raft_consensus.cc:2258] T 977b373d3d134811a0ac51b31ca0c377 P e4133e575b754c4fb437792fca3afd09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:40:31.952164 2040 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:40:31.953063 2040 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:40:31.953254 3045 ts_tablet_manager.cc:1905] T 977b373d3d134811a0ac51b31ca0c377 P e4133e575b754c4fb437792fca3afd09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:40:31.959884 3050 tablet_replica.cc:330] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7: stopping tablet replica
I20260811 18:40:31.960533 3050 raft_consensus.cc:2229] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:40:31.961895 3050 raft_consensus.cc:2258] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:40:31.962018 3045 ts_tablet_manager.cc:1918] T 977b373d3d134811a0ac51b31ca0c377 P e4133e575b754c4fb437792fca3afd09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:40:31.962211 3045 log.cc:1196] T 977b373d3d134811a0ac51b31ca0c377 P e4133e575b754c4fb437792fca3afd09: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/977b373d3d134811a0ac51b31ca0c377
I20260811 18:40:31.962452 3046 tablet_replica.cc:330] T 977b373d3d134811a0ac51b31ca0c377 P 7209845a058349c6a108c7fa04c5b580: stopping tablet replica
I20260811 18:40:31.963281 3046 raft_consensus.cc:2229] T 977b373d3d134811a0ac51b31ca0c377 P 7209845a058349c6a108c7fa04c5b580 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:40:31.963450 2289 tablet_service.cc:1467] Processing CreateTablet for tablet 37f2735356a24cada6638038f0dc65b0 (DEFAULT_TABLE table=test-table [id=5fbf7995ed6643e8a57df26e41c44ac0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:31.964095 2794 tablet_service.cc:1467] Processing CreateTablet for tablet 37f2735356a24cada6638038f0dc65b0 (DEFAULT_TABLE table=test-table [id=5fbf7995ed6643e8a57df26e41c44ac0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:31.964622 2289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37f2735356a24cada6638038f0dc65b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:31.964681 2794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37f2735356a24cada6638038f0dc65b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:31.965926 3046 raft_consensus.cc:2258] T 977b373d3d134811a0ac51b31ca0c377 P 7209845a058349c6a108c7fa04c5b580 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:40:31.966952 3050 ts_tablet_manager.cc:1905] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:40:31.971493 2543 tablet_service.cc:1467] Processing CreateTablet for tablet 37f2735356a24cada6638038f0dc65b0 (DEFAULT_TABLE table=test-table [id=5fbf7995ed6643e8a57df26e41c44ac0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:31.974599 3046 ts_tablet_manager.cc:1905] T 977b373d3d134811a0ac51b31ca0c377 P 7209845a058349c6a108c7fa04c5b580: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:40:31.975714 2961 tablet_bootstrap.cc:492] T 37f2735356a24cada6638038f0dc65b0 P e4133e575b754c4fb437792fca3afd09: Bootstrap starting.
I20260811 18:40:31.975993 2543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37f2735356a24cada6638038f0dc65b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:31.978044 2961 tablet_bootstrap.cc:654] T 37f2735356a24cada6638038f0dc65b0 P e4133e575b754c4fb437792fca3afd09: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:31.981635 3045 ts_tablet_manager.cc:1939] T 977b373d3d134811a0ac51b31ca0c377 P e4133e575b754c4fb437792fca3afd09: Deleting consensus metadata
I20260811 18:40:31.984129 2015 catalog_manager.cc:4841] TS e4133e575b754c4fb437792fca3afd09 (127.30.185.131:34685): tablet 977b373d3d134811a0ac51b31ca0c377 (table test-table [id=48dd3bd7e805475bb64aaa811e528a02]) successfully deleted
I20260811 18:40:31.984632 2964 tablet_bootstrap.cc:492] T 37f2735356a24cada6638038f0dc65b0 P 7209845a058349c6a108c7fa04c5b580: Bootstrap starting.
I20260811 18:40:31.986681 3050 ts_tablet_manager.cc:1918] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:40:31.988103 2964 tablet_bootstrap.cc:654] T 37f2735356a24cada6638038f0dc65b0 P 7209845a058349c6a108c7fa04c5b580: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:31.992604 2961 tablet_bootstrap.cc:492] T 37f2735356a24cada6638038f0dc65b0 P e4133e575b754c4fb437792fca3afd09: No bootstrap required, opened a new log
I20260811 18:40:31.992990 2961 ts_tablet_manager.cc:1397] T 37f2735356a24cada6638038f0dc65b0 P e4133e575b754c4fb437792fca3afd09: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.008s
I20260811 18:40:31.994145 2961 raft_consensus.cc:348] T 37f2735356a24cada6638038f0dc65b0 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:31.992986 3050 log.cc:1196] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/977b373d3d134811a0ac51b31ca0c377
I20260811 18:40:31.994772 2961 raft_consensus.cc:374] T 37f2735356a24cada6638038f0dc65b0 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:31.995025 2961 raft_consensus.cc:729] T 37f2735356a24cada6638038f0dc65b0 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4133e575b754c4fb437792fca3afd09, State: Initialized, Role: FOLLOWER
I20260811 18:40:31.995205 3050 ts_tablet_manager.cc:1939] T 977b373d3d134811a0ac51b31ca0c377 P a1e13f628490435d85fa802f60e2d0f7: Deleting consensus metadata
I20260811 18:40:31.996049 2961 consensus_queue.cc:260] T 37f2735356a24cada6638038f0dc65b0 P e4133e575b754c4fb437792fca3afd09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:31.996889 2966 tablet_bootstrap.cc:492] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7: Bootstrap starting.
I20260811 18:40:31.998313 2964 tablet_bootstrap.cc:492] T 37f2735356a24cada6638038f0dc65b0 P 7209845a058349c6a108c7fa04c5b580: No bootstrap required, opened a new log
I20260811 18:40:31.998591 2964 ts_tablet_manager.cc:1397] T 37f2735356a24cada6638038f0dc65b0 P 7209845a058349c6a108c7fa04c5b580: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.003s
I20260811 18:40:31.998909 2966 tablet_bootstrap.cc:654] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:31.999588 2964 raft_consensus.cc:348] T 37f2735356a24cada6638038f0dc65b0 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:31.999908 2964 raft_consensus.cc:374] T 37f2735356a24cada6638038f0dc65b0 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:32.000012 2964 raft_consensus.cc:729] T 37f2735356a24cada6638038f0dc65b0 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7209845a058349c6a108c7fa04c5b580, State: Initialized, Role: FOLLOWER
I20260811 18:40:32.000231 2961 ts_tablet_manager.cc:1428] T 37f2735356a24cada6638038f0dc65b0 P e4133e575b754c4fb437792fca3afd09: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260811 18:40:32.000278 2964 consensus_queue.cc:260] T 37f2735356a24cada6638038f0dc65b0 P 7209845a058349c6a108c7fa04c5b580 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:32.000932 2964 ts_tablet_manager.cc:1428] T 37f2735356a24cada6638038f0dc65b0 P 7209845a058349c6a108c7fa04c5b580: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:40:32.004005 3046 ts_tablet_manager.cc:1918] T 977b373d3d134811a0ac51b31ca0c377 P 7209845a058349c6a108c7fa04c5b580: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:40:32.005261 2015 catalog_manager.cc:4841] TS a1e13f628490435d85fa802f60e2d0f7 (127.30.185.130:44815): tablet 977b373d3d134811a0ac51b31ca0c377 (table test-table [id=48dd3bd7e805475bb64aaa811e528a02]) successfully deleted
I20260811 18:40:32.004289 3046 log.cc:1196] T 977b373d3d134811a0ac51b31ca0c377 P 7209845a058349c6a108c7fa04c5b580: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-0/wal/wals/977b373d3d134811a0ac51b31ca0c377
I20260811 18:40:32.008244 3046 ts_tablet_manager.cc:1939] T 977b373d3d134811a0ac51b31ca0c377 P 7209845a058349c6a108c7fa04c5b580: Deleting consensus metadata
I20260811 18:40:32.009279 2966 tablet_bootstrap.cc:492] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7: No bootstrap required, opened a new log
I20260811 18:40:32.009584 2966 ts_tablet_manager.cc:1397] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.004s
I20260811 18:40:32.010488 2018 catalog_manager.cc:4841] TS 7209845a058349c6a108c7fa04c5b580 (127.30.185.129:45641): tablet 977b373d3d134811a0ac51b31ca0c377 (table test-table [id=48dd3bd7e805475bb64aaa811e528a02]) successfully deleted
I20260811 18:40:32.010732 2966 raft_consensus.cc:348] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:32.011051 2966 raft_consensus.cc:374] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:32.011169 2966 raft_consensus.cc:729] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1e13f628490435d85fa802f60e2d0f7, State: Initialized, Role: FOLLOWER
I20260811 18:40:32.011493 2966 consensus_queue.cc:260] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:32.012091 2966 ts_tablet_manager.cc:1428] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:40:32.096832 2981 raft_consensus.cc:482] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:40:32.097278 2981 raft_consensus.cc:504] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:32.098064 2981 leader_election.cc:288] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7209845a058349c6a108c7fa04c5b580 (127.30.185.129:45641), e4133e575b754c4fb437792fca3afd09 (127.30.185.131:34685)
I20260811 18:40:32.098706 2322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37f2735356a24cada6638038f0dc65b0" candidate_uuid: "a1e13f628490435d85fa802f60e2d0f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7209845a058349c6a108c7fa04c5b580" is_pre_election: true
I20260811 18:40:32.098824 2833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37f2735356a24cada6638038f0dc65b0" candidate_uuid: "a1e13f628490435d85fa802f60e2d0f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4133e575b754c4fb437792fca3afd09" is_pre_election: true
I20260811 18:40:32.099056 2833 raft_consensus.cc:2454] T 37f2735356a24cada6638038f0dc65b0 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1e13f628490435d85fa802f60e2d0f7 in term 0.
I20260811 18:40:32.099054 2322 raft_consensus.cc:2454] T 37f2735356a24cada6638038f0dc65b0 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1e13f628490435d85fa802f60e2d0f7 in term 0.
I20260811 18:40:32.099540 2450 leader_election.cc:302] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1e13f628490435d85fa802f60e2d0f7, e4133e575b754c4fb437792fca3afd09; no voters:
I20260811 18:40:32.099944 2981 raft_consensus.cc:2781] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:40:32.100075 2981 raft_consensus.cc:482] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:40:32.100253 2981 raft_consensus.cc:3037] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:32.103011 2981 raft_consensus.cc:504] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:32.104173 2322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37f2735356a24cada6638038f0dc65b0" candidate_uuid: "a1e13f628490435d85fa802f60e2d0f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7209845a058349c6a108c7fa04c5b580"
I20260811 18:40:32.104166 2833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37f2735356a24cada6638038f0dc65b0" candidate_uuid: "a1e13f628490435d85fa802f60e2d0f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4133e575b754c4fb437792fca3afd09"
I20260811 18:40:32.104431 2322 raft_consensus.cc:3037] T 37f2735356a24cada6638038f0dc65b0 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:32.104432 2833 raft_consensus.cc:3037] T 37f2735356a24cada6638038f0dc65b0 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:32.107951 2833 raft_consensus.cc:2454] T 37f2735356a24cada6638038f0dc65b0 P e4133e575b754c4fb437792fca3afd09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1e13f628490435d85fa802f60e2d0f7 in term 1.
I20260811 18:40:32.108605 2450 leader_election.cc:302] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1e13f628490435d85fa802f60e2d0f7, e4133e575b754c4fb437792fca3afd09; no voters:
I20260811 18:40:32.108927 2322 raft_consensus.cc:2454] T 37f2735356a24cada6638038f0dc65b0 P 7209845a058349c6a108c7fa04c5b580 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1e13f628490435d85fa802f60e2d0f7 in term 1.
I20260811 18:40:32.109072 2973 raft_consensus.cc:2781] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:40:32.109396 2973 raft_consensus.cc:686] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7 [term 1 LEADER]: Becoming Leader. State: Replica: a1e13f628490435d85fa802f60e2d0f7, State: Running, Role: LEADER
I20260811 18:40:32.109794 2973 consensus_queue.cc:237] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:32.110308 2981 leader_election.cc:288] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7 [CANDIDATE]: Term 1 election: Requested vote from peers 7209845a058349c6a108c7fa04c5b580 (127.30.185.129:45641), e4133e575b754c4fb437792fca3afd09 (127.30.185.131:34685)
I20260811 18:40:32.113500 2040 catalog_manager.cc:5495] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1e13f628490435d85fa802f60e2d0f7 (127.30.185.130). New cstate: current_term: 1 leader_uuid: "a1e13f628490435d85fa802f60e2d0f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } health_report { overall_health: UNKNOWN } } }
I20260811 18:40:32.157220 2322 raft_consensus.cc:1261] T 37f2735356a24cada6638038f0dc65b0 P 7209845a058349c6a108c7fa04c5b580 [term 1 FOLLOWER]: Refusing update from remote peer a1e13f628490435d85fa802f60e2d0f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:40:32.158646 2981 consensus_queue.cc:1035] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:40:32.161409 2833 raft_consensus.cc:1261] T 37f2735356a24cada6638038f0dc65b0 P e4133e575b754c4fb437792fca3afd09 [term 1 FOLLOWER]: Refusing update from remote peer a1e13f628490435d85fa802f60e2d0f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:40:32.162144 2981 consensus_queue.cc:1035] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:40:32.194689 3011 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 18:40:33.816008 2043 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:40:33.816324 2043 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:40:33.820588 2043 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435: Sending DeleteTablet for 3 replicas of tablet 37f2735356a24cada6638038f0dc65b0
I20260811 18:40:33.821627 2289 tablet_service.cc:1514] Processing DeleteTablet for tablet 37f2735356a24cada6638038f0dc65b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:33 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:34644
I20260811 18:40:33.821751 2543 tablet_service.cc:1514] Processing DeleteTablet for tablet 37f2735356a24cada6638038f0dc65b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:33 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:56268
I20260811 18:40:33.821848 2794 tablet_service.cc:1514] Processing DeleteTablet for tablet 37f2735356a24cada6638038f0dc65b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:33 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:56058
I20260811 18:40:33.834384 3114 tablet_replica.cc:330] T 37f2735356a24cada6638038f0dc65b0 P 7209845a058349c6a108c7fa04c5b580: stopping tablet replica
I20260811 18:40:33.835135 3114 raft_consensus.cc:2229] T 37f2735356a24cada6638038f0dc65b0 P 7209845a058349c6a108c7fa04c5b580 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:40:33.835527 3114 raft_consensus.cc:2258] T 37f2735356a24cada6638038f0dc65b0 P 7209845a058349c6a108c7fa04c5b580 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:40:33.835573 3118 tablet_replica.cc:330] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7: stopping tablet replica
I20260811 18:40:33.835887 3120 tablet_replica.cc:330] T 37f2735356a24cada6638038f0dc65b0 P e4133e575b754c4fb437792fca3afd09: stopping tablet replica
I20260811 18:40:33.836223 3120 raft_consensus.cc:2229] T 37f2735356a24cada6638038f0dc65b0 P e4133e575b754c4fb437792fca3afd09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:40:33.836540 3120 raft_consensus.cc:2258] T 37f2735356a24cada6638038f0dc65b0 P e4133e575b754c4fb437792fca3afd09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:40:33.836604 3118 raft_consensus.cc:2229] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:40:33.836740 3114 ts_tablet_manager.cc:1905] T 37f2735356a24cada6638038f0dc65b0 P 7209845a058349c6a108c7fa04c5b580: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:40:33.837218 3118 raft_consensus.cc:2258] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:40:33.838265 3118 ts_tablet_manager.cc:1905] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:40:33.838368 3120 ts_tablet_manager.cc:1905] T 37f2735356a24cada6638038f0dc65b0 P e4133e575b754c4fb437792fca3afd09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:40:33.843767 3114 ts_tablet_manager.cc:1918] T 37f2735356a24cada6638038f0dc65b0 P 7209845a058349c6a108c7fa04c5b580: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:40:33.843940 3114 log.cc:1196] T 37f2735356a24cada6638038f0dc65b0 P 7209845a058349c6a108c7fa04c5b580: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-0/wal/wals/37f2735356a24cada6638038f0dc65b0
I20260811 18:40:33.844435 3114 ts_tablet_manager.cc:1939] T 37f2735356a24cada6638038f0dc65b0 P 7209845a058349c6a108c7fa04c5b580: Deleting consensus metadata
I20260811 18:40:33.845405 3118 ts_tablet_manager.cc:1918] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:40:33.845559 3118 log.cc:1196] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/37f2735356a24cada6638038f0dc65b0
I20260811 18:40:33.845707 3120 ts_tablet_manager.cc:1918] T 37f2735356a24cada6638038f0dc65b0 P e4133e575b754c4fb437792fca3afd09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:40:33.845747 2018 catalog_manager.cc:4841] TS 7209845a058349c6a108c7fa04c5b580 (127.30.185.129:45641): tablet 37f2735356a24cada6638038f0dc65b0 (table test-table [id=5fbf7995ed6643e8a57df26e41c44ac0]) successfully deleted
I20260811 18:40:33.845885 3120 log.cc:1196] T 37f2735356a24cada6638038f0dc65b0 P e4133e575b754c4fb437792fca3afd09: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/37f2735356a24cada6638038f0dc65b0
I20260811 18:40:33.846057 3118 ts_tablet_manager.cc:1939] T 37f2735356a24cada6638038f0dc65b0 P a1e13f628490435d85fa802f60e2d0f7: Deleting consensus metadata
I20260811 18:40:33.846364 3120 ts_tablet_manager.cc:1939] T 37f2735356a24cada6638038f0dc65b0 P e4133e575b754c4fb437792fca3afd09: Deleting consensus metadata
I20260811 18:40:33.847157 2015 catalog_manager.cc:4841] TS a1e13f628490435d85fa802f60e2d0f7 (127.30.185.130:44815): tablet 37f2735356a24cada6638038f0dc65b0 (table test-table [id=5fbf7995ed6643e8a57df26e41c44ac0]) successfully deleted
I20260811 18:40:33.847450 2015 catalog_manager.cc:4841] TS e4133e575b754c4fb437792fca3afd09 (127.30.185.131:34685): tablet 37f2735356a24cada6638038f0dc65b0 (table test-table [id=5fbf7995ed6643e8a57df26e41c44ac0]) successfully deleted
Aug 11 18:40:33 dist-test-slave-z8x9 krb5kdc[1968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473633, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:40:33 dist-test-slave-z8x9 krb5kdc[1968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473633, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:33.897943 3135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:33.882334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44094 (local address 127.30.185.190:42283)
0811 18:40:33.882976 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:33.882982 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:33.883005 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:40:33.883136 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:33.883140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:33.883208 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:33.883327 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:33.884699 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:33.885645 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:33.886992 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:33.887422 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:33.887534 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:33.891718 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:33.891724 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:33.891728 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:33.891767 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:33.895033 (+ 3266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:33.895725 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:33.895729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:33.895732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:33.895888 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:33.896360 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:33.896365 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:33.896367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:33.896718 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:33.896981 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:33.897348 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 18:40:33.897584 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":313,"threads_started":1}
I20260811 18:40:33.901693 2043 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44094:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:40:33.902354 2043 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:40:33.909001 2289 tablet_service.cc:1467] Processing CreateTablet for tablet 69ec965881eb4c57b05188319a5bd9b5 (DEFAULT_TABLE table=test-table [id=a8e94cf234ea4d93aed1fbd7f907a03a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:33.909569 2289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69ec965881eb4c57b05188319a5bd9b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:33.909509 2543 tablet_service.cc:1467] Processing CreateTablet for tablet 69ec965881eb4c57b05188319a5bd9b5 (DEFAULT_TABLE table=test-table [id=a8e94cf234ea4d93aed1fbd7f907a03a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:33.909586 2794 tablet_service.cc:1467] Processing CreateTablet for tablet 69ec965881eb4c57b05188319a5bd9b5 (DEFAULT_TABLE table=test-table [id=a8e94cf234ea4d93aed1fbd7f907a03a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:33.909981 2794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69ec965881eb4c57b05188319a5bd9b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:33.910003 2543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69ec965881eb4c57b05188319a5bd9b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:33.918555 3142 tablet_bootstrap.cc:492] T 69ec965881eb4c57b05188319a5bd9b5 P 7209845a058349c6a108c7fa04c5b580: Bootstrap starting.
I20260811 18:40:33.921293 3142 tablet_bootstrap.cc:654] T 69ec965881eb4c57b05188319a5bd9b5 P 7209845a058349c6a108c7fa04c5b580: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:33.924224 3143 tablet_bootstrap.cc:492] T 69ec965881eb4c57b05188319a5bd9b5 P e4133e575b754c4fb437792fca3afd09: Bootstrap starting.
I20260811 18:40:33.926368 3143 tablet_bootstrap.cc:654] T 69ec965881eb4c57b05188319a5bd9b5 P e4133e575b754c4fb437792fca3afd09: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:33.927336 3141 tablet_bootstrap.cc:492] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7: Bootstrap starting.
I20260811 18:40:33.929345 3142 tablet_bootstrap.cc:492] T 69ec965881eb4c57b05188319a5bd9b5 P 7209845a058349c6a108c7fa04c5b580: No bootstrap required, opened a new log
I20260811 18:40:33.929363 3141 tablet_bootstrap.cc:654] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:33.929529 3142 ts_tablet_manager.cc:1397] T 69ec965881eb4c57b05188319a5bd9b5 P 7209845a058349c6a108c7fa04c5b580: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260811 18:40:33.930248 3142 raft_consensus.cc:348] T 69ec965881eb4c57b05188319a5bd9b5 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:33.930510 3142 raft_consensus.cc:374] T 69ec965881eb4c57b05188319a5bd9b5 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:33.930603 3142 raft_consensus.cc:729] T 69ec965881eb4c57b05188319a5bd9b5 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7209845a058349c6a108c7fa04c5b580, State: Initialized, Role: FOLLOWER
I20260811 18:40:33.930903 3142 consensus_queue.cc:260] T 69ec965881eb4c57b05188319a5bd9b5 P 7209845a058349c6a108c7fa04c5b580 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:33.931507 3142 ts_tablet_manager.cc:1428] T 69ec965881eb4c57b05188319a5bd9b5 P 7209845a058349c6a108c7fa04c5b580: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:40:33.934743 3141 tablet_bootstrap.cc:492] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7: No bootstrap required, opened a new log
I20260811 18:40:33.934916 3141 ts_tablet_manager.cc:1397] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260811 18:40:33.935837 3143 tablet_bootstrap.cc:492] T 69ec965881eb4c57b05188319a5bd9b5 P e4133e575b754c4fb437792fca3afd09: No bootstrap required, opened a new log
I20260811 18:40:33.936008 3143 ts_tablet_manager.cc:1397] T 69ec965881eb4c57b05188319a5bd9b5 P e4133e575b754c4fb437792fca3afd09: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.002s
I20260811 18:40:33.935885 3141 raft_consensus.cc:348] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:33.936184 3141 raft_consensus.cc:374] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:33.936292 3141 raft_consensus.cc:729] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1e13f628490435d85fa802f60e2d0f7, State: Initialized, Role: FOLLOWER
I20260811 18:40:33.936666 3141 consensus_queue.cc:260] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:33.936947 3143 raft_consensus.cc:348] T 69ec965881eb4c57b05188319a5bd9b5 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:33.937282 3143 raft_consensus.cc:374] T 69ec965881eb4c57b05188319a5bd9b5 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:33.937366 3141 ts_tablet_manager.cc:1428] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 18:40:33.937394 3143 raft_consensus.cc:729] T 69ec965881eb4c57b05188319a5bd9b5 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4133e575b754c4fb437792fca3afd09, State: Initialized, Role: FOLLOWER
I20260811 18:40:33.937633 3143 consensus_queue.cc:260] T 69ec965881eb4c57b05188319a5bd9b5 P e4133e575b754c4fb437792fca3afd09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:33.938565 3143 ts_tablet_manager.cc:1428] T 69ec965881eb4c57b05188319a5bd9b5 P e4133e575b754c4fb437792fca3afd09: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 18:40:34.065351 3084 raft_consensus.cc:482] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:40:34.065552 3084 raft_consensus.cc:504] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:34.066416 3084 leader_election.cc:288] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7209845a058349c6a108c7fa04c5b580 (127.30.185.129:45641), e4133e575b754c4fb437792fca3afd09 (127.30.185.131:34685)
I20260811 18:40:34.067114 2829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69ec965881eb4c57b05188319a5bd9b5" candidate_uuid: "a1e13f628490435d85fa802f60e2d0f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4133e575b754c4fb437792fca3afd09" is_pre_election: true
I20260811 18:40:34.067106 2322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69ec965881eb4c57b05188319a5bd9b5" candidate_uuid: "a1e13f628490435d85fa802f60e2d0f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7209845a058349c6a108c7fa04c5b580" is_pre_election: true
I20260811 18:40:34.067404 2829 raft_consensus.cc:2454] T 69ec965881eb4c57b05188319a5bd9b5 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1e13f628490435d85fa802f60e2d0f7 in term 0.
I20260811 18:40:34.067404 2322 raft_consensus.cc:2454] T 69ec965881eb4c57b05188319a5bd9b5 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1e13f628490435d85fa802f60e2d0f7 in term 0.
I20260811 18:40:34.067916 2452 leader_election.cc:302] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7209845a058349c6a108c7fa04c5b580, a1e13f628490435d85fa802f60e2d0f7, e4133e575b754c4fb437792fca3afd09; no voters:
I20260811 18:40:34.068214 3084 raft_consensus.cc:2781] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:40:34.068327 3084 raft_consensus.cc:482] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:40:34.068454 3084 raft_consensus.cc:3037] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:34.070994 3084 raft_consensus.cc:504] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:34.071667 3084 leader_election.cc:288] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7 [CANDIDATE]: Term 1 election: Requested vote from peers 7209845a058349c6a108c7fa04c5b580 (127.30.185.129:45641), e4133e575b754c4fb437792fca3afd09 (127.30.185.131:34685)
I20260811 18:40:34.072165 2322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69ec965881eb4c57b05188319a5bd9b5" candidate_uuid: "a1e13f628490435d85fa802f60e2d0f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7209845a058349c6a108c7fa04c5b580"
I20260811 18:40:34.072180 2829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69ec965881eb4c57b05188319a5bd9b5" candidate_uuid: "a1e13f628490435d85fa802f60e2d0f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4133e575b754c4fb437792fca3afd09"
I20260811 18:40:34.072430 2322 raft_consensus.cc:3037] T 69ec965881eb4c57b05188319a5bd9b5 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:34.072430 2829 raft_consensus.cc:3037] T 69ec965881eb4c57b05188319a5bd9b5 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:34.075317 2322 raft_consensus.cc:2454] T 69ec965881eb4c57b05188319a5bd9b5 P 7209845a058349c6a108c7fa04c5b580 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1e13f628490435d85fa802f60e2d0f7 in term 1.
I20260811 18:40:34.075595 2829 raft_consensus.cc:2454] T 69ec965881eb4c57b05188319a5bd9b5 P e4133e575b754c4fb437792fca3afd09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1e13f628490435d85fa802f60e2d0f7 in term 1.
I20260811 18:40:34.075793 2452 leader_election.cc:302] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7209845a058349c6a108c7fa04c5b580, a1e13f628490435d85fa802f60e2d0f7; no voters:
I20260811 18:40:34.076123 3084 raft_consensus.cc:2781] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:40:34.076278 3084 raft_consensus.cc:686] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7 [term 1 LEADER]: Becoming Leader. State: Replica: a1e13f628490435d85fa802f60e2d0f7, State: Running, Role: LEADER
I20260811 18:40:34.076576 3084 consensus_queue.cc:237] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:34.079327 2043 catalog_manager.cc:5495] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1e13f628490435d85fa802f60e2d0f7 (127.30.185.130). New cstate: current_term: 1 leader_uuid: "a1e13f628490435d85fa802f60e2d0f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } health_report { overall_health: UNKNOWN } } }
I20260811 18:40:34.139772 3157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:34.131545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60142 (local address 127.30.185.130:44815)
0811 18:40:34.134751 (+ 3206us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:34.134755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:34.134775 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:40:34.134832 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:34.134836 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:34.134902 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:34.134992 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:40:34.135736 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:34.136545 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:34.138022 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:34.138436 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:34.138533 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:34.139023 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:40:34.139129 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:40:34.139197 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:34.139342 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 18:40:34.139400 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3087,"thread_start_us":2952,"threads_started":1}
I20260811 18:40:34.149451 2322 raft_consensus.cc:1261] T 69ec965881eb4c57b05188319a5bd9b5 P 7209845a058349c6a108c7fa04c5b580 [term 1 FOLLOWER]: Refusing update from remote peer a1e13f628490435d85fa802f60e2d0f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:40:34.149448 2829 raft_consensus.cc:1261] T 69ec965881eb4c57b05188319a5bd9b5 P e4133e575b754c4fb437792fca3afd09 [term 1 FOLLOWER]: Refusing update from remote peer a1e13f628490435d85fa802f60e2d0f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:40:34.150082 3040 consensus_queue.cc:1035] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:40:34.150519 3084 consensus_queue.cc:1035] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:40:34.200321 2043 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:40:34.200613 2043 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:40:34.203508 2043 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435: Sending DeleteTablet for 3 replicas of tablet 69ec965881eb4c57b05188319a5bd9b5
I20260811 18:40:34.204308 2289 tablet_service.cc:1514] Processing DeleteTablet for tablet 69ec965881eb4c57b05188319a5bd9b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:34 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:34644
I20260811 18:40:34.204468 2543 tablet_service.cc:1514] Processing DeleteTablet for tablet 69ec965881eb4c57b05188319a5bd9b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:34 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:56268
I20260811 18:40:34.204546 2794 tablet_service.cc:1514] Processing DeleteTablet for tablet 69ec965881eb4c57b05188319a5bd9b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:34 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:56058
I20260811 18:40:34.204631 3114 tablet_replica.cc:330] T 69ec965881eb4c57b05188319a5bd9b5 P 7209845a058349c6a108c7fa04c5b580: stopping tablet replica
I20260811 18:40:34.204833 3120 tablet_replica.cc:330] T 69ec965881eb4c57b05188319a5bd9b5 P e4133e575b754c4fb437792fca3afd09: stopping tablet replica
I20260811 18:40:34.204900 3118 tablet_replica.cc:330] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7: stopping tablet replica
I20260811 18:40:34.204867 3114 raft_consensus.cc:2229] T 69ec965881eb4c57b05188319a5bd9b5 P 7209845a058349c6a108c7fa04c5b580 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:40:34.205032 3120 raft_consensus.cc:2229] T 69ec965881eb4c57b05188319a5bd9b5 P e4133e575b754c4fb437792fca3afd09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:40:34.205072 3114 raft_consensus.cc:2258] T 69ec965881eb4c57b05188319a5bd9b5 P 7209845a058349c6a108c7fa04c5b580 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:40:34.205083 3118 raft_consensus.cc:2229] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:40:34.205314 3120 raft_consensus.cc:2258] T 69ec965881eb4c57b05188319a5bd9b5 P e4133e575b754c4fb437792fca3afd09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:40:34.205446 3118 raft_consensus.cc:2258] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:40:34.205788 2043 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44094:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:40:34.206409 2043 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:40:34.206516 3120 ts_tablet_manager.cc:1905] T 69ec965881eb4c57b05188319a5bd9b5 P e4133e575b754c4fb437792fca3afd09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:40:34.206646 3114 ts_tablet_manager.cc:1905] T 69ec965881eb4c57b05188319a5bd9b5 P 7209845a058349c6a108c7fa04c5b580: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:40:34.206754 3118 ts_tablet_manager.cc:1905] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:40:34.212179 3120 ts_tablet_manager.cc:1918] T 69ec965881eb4c57b05188319a5bd9b5 P e4133e575b754c4fb437792fca3afd09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:40:34.212388 3120 log.cc:1196] T 69ec965881eb4c57b05188319a5bd9b5 P e4133e575b754c4fb437792fca3afd09: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/69ec965881eb4c57b05188319a5bd9b5
I20260811 18:40:34.212837 3120 ts_tablet_manager.cc:1939] T 69ec965881eb4c57b05188319a5bd9b5 P e4133e575b754c4fb437792fca3afd09: Deleting consensus metadata
I20260811 18:40:34.213274 3114 ts_tablet_manager.cc:1918] T 69ec965881eb4c57b05188319a5bd9b5 P 7209845a058349c6a108c7fa04c5b580: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:40:34.213476 3114 log.cc:1196] T 69ec965881eb4c57b05188319a5bd9b5 P 7209845a058349c6a108c7fa04c5b580: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-0/wal/wals/69ec965881eb4c57b05188319a5bd9b5
I20260811 18:40:34.213937 3114 ts_tablet_manager.cc:1939] T 69ec965881eb4c57b05188319a5bd9b5 P 7209845a058349c6a108c7fa04c5b580: Deleting consensus metadata
I20260811 18:40:34.213989 3118 ts_tablet_manager.cc:1918] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:40:34.213994 2015 catalog_manager.cc:4841] TS e4133e575b754c4fb437792fca3afd09 (127.30.185.131:34685): tablet 69ec965881eb4c57b05188319a5bd9b5 (table test-table [id=a8e94cf234ea4d93aed1fbd7f907a03a]) successfully deleted
I20260811 18:40:34.214149 3118 log.cc:1196] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/69ec965881eb4c57b05188319a5bd9b5
I20260811 18:40:34.214591 3118 ts_tablet_manager.cc:1939] T 69ec965881eb4c57b05188319a5bd9b5 P a1e13f628490435d85fa802f60e2d0f7: Deleting consensus metadata
I20260811 18:40:34.215312 2018 catalog_manager.cc:4841] TS 7209845a058349c6a108c7fa04c5b580 (127.30.185.129:45641): tablet 69ec965881eb4c57b05188319a5bd9b5 (table test-table [id=a8e94cf234ea4d93aed1fbd7f907a03a]) successfully deleted
I20260811 18:40:34.216192 2015 catalog_manager.cc:4841] TS a1e13f628490435d85fa802f60e2d0f7 (127.30.185.130:44815): tablet 69ec965881eb4c57b05188319a5bd9b5 (table test-table [id=a8e94cf234ea4d93aed1fbd7f907a03a]) successfully deleted
I20260811 18:40:34.217007 2543 tablet_service.cc:1467] Processing CreateTablet for tablet 332701e1cc194d02a070ca30af47d4b8 (DEFAULT_TABLE table=test-table [id=4d5868b58aed4cb8923b3a1bf48fae37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:34.216984 2289 tablet_service.cc:1467] Processing CreateTablet for tablet 332701e1cc194d02a070ca30af47d4b8 (DEFAULT_TABLE table=test-table [id=4d5868b58aed4cb8923b3a1bf48fae37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:34.217031 2794 tablet_service.cc:1467] Processing CreateTablet for tablet 332701e1cc194d02a070ca30af47d4b8 (DEFAULT_TABLE table=test-table [id=4d5868b58aed4cb8923b3a1bf48fae37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:34.217471 2543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 332701e1cc194d02a070ca30af47d4b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:34.217476 2289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 332701e1cc194d02a070ca30af47d4b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:34.217494 2794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 332701e1cc194d02a070ca30af47d4b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:34.222422 3141 tablet_bootstrap.cc:492] T 332701e1cc194d02a070ca30af47d4b8 P a1e13f628490435d85fa802f60e2d0f7: Bootstrap starting.
I20260811 18:40:34.223989 3141 tablet_bootstrap.cc:654] T 332701e1cc194d02a070ca30af47d4b8 P a1e13f628490435d85fa802f60e2d0f7: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:34.224434 3142 tablet_bootstrap.cc:492] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580: Bootstrap starting.
I20260811 18:40:34.224466 3143 tablet_bootstrap.cc:492] T 332701e1cc194d02a070ca30af47d4b8 P e4133e575b754c4fb437792fca3afd09: Bootstrap starting.
I20260811 18:40:34.226214 3142 tablet_bootstrap.cc:654] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:34.226320 3143 tablet_bootstrap.cc:654] T 332701e1cc194d02a070ca30af47d4b8 P e4133e575b754c4fb437792fca3afd09: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:34.226562 3141 tablet_bootstrap.cc:492] T 332701e1cc194d02a070ca30af47d4b8 P a1e13f628490435d85fa802f60e2d0f7: No bootstrap required, opened a new log
I20260811 18:40:34.226742 3141 ts_tablet_manager.cc:1397] T 332701e1cc194d02a070ca30af47d4b8 P a1e13f628490435d85fa802f60e2d0f7: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 18:40:34.227609 3141 raft_consensus.cc:348] T 332701e1cc194d02a070ca30af47d4b8 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:34.227892 3141 raft_consensus.cc:374] T 332701e1cc194d02a070ca30af47d4b8 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:34.228019 3141 raft_consensus.cc:729] T 332701e1cc194d02a070ca30af47d4b8 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1e13f628490435d85fa802f60e2d0f7, State: Initialized, Role: FOLLOWER
I20260811 18:40:34.228394 3142 tablet_bootstrap.cc:492] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580: No bootstrap required, opened a new log
I20260811 18:40:34.228294 3141 consensus_queue.cc:260] T 332701e1cc194d02a070ca30af47d4b8 P a1e13f628490435d85fa802f60e2d0f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:34.228600 3142 ts_tablet_manager.cc:1397] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 18:40:34.228884 3141 ts_tablet_manager.cc:1428] T 332701e1cc194d02a070ca30af47d4b8 P a1e13f628490435d85fa802f60e2d0f7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:40:34.229394 3142 raft_consensus.cc:348] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:34.229662 3142 raft_consensus.cc:374] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:34.229759 3142 raft_consensus.cc:729] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7209845a058349c6a108c7fa04c5b580, State: Initialized, Role: FOLLOWER
I20260811 18:40:34.230062 3143 tablet_bootstrap.cc:492] T 332701e1cc194d02a070ca30af47d4b8 P e4133e575b754c4fb437792fca3afd09: No bootstrap required, opened a new log
I20260811 18:40:34.229990 3142 consensus_queue.cc:260] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:34.230180 3143 ts_tablet_manager.cc:1397] T 332701e1cc194d02a070ca30af47d4b8 P e4133e575b754c4fb437792fca3afd09: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260811 18:40:34.230499 3142 ts_tablet_manager.cc:1428] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:40:34.230695 3143 raft_consensus.cc:348] T 332701e1cc194d02a070ca30af47d4b8 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:34.230924 3143 raft_consensus.cc:374] T 332701e1cc194d02a070ca30af47d4b8 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:34.231009 3143 raft_consensus.cc:729] T 332701e1cc194d02a070ca30af47d4b8 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4133e575b754c4fb437792fca3afd09, State: Initialized, Role: FOLLOWER
I20260811 18:40:34.231256 3143 consensus_queue.cc:260] T 332701e1cc194d02a070ca30af47d4b8 P e4133e575b754c4fb437792fca3afd09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:34.231832 3143 ts_tablet_manager.cc:1428] T 332701e1cc194d02a070ca30af47d4b8 P e4133e575b754c4fb437792fca3afd09: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 18:40:34.326010 3096 raft_consensus.cc:482] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:40:34.326298 3096 raft_consensus.cc:504] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:34.327036 3096 leader_election.cc:288] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1e13f628490435d85fa802f60e2d0f7 (127.30.185.130:44815), e4133e575b754c4fb437792fca3afd09 (127.30.185.131:34685)
I20260811 18:40:34.327641 2587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "332701e1cc194d02a070ca30af47d4b8" candidate_uuid: "7209845a058349c6a108c7fa04c5b580" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1e13f628490435d85fa802f60e2d0f7" is_pre_election: true
I20260811 18:40:34.327684 2833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "332701e1cc194d02a070ca30af47d4b8" candidate_uuid: "7209845a058349c6a108c7fa04c5b580" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4133e575b754c4fb437792fca3afd09" is_pre_election: true
I20260811 18:40:34.327903 2587 raft_consensus.cc:2454] T 332701e1cc194d02a070ca30af47d4b8 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7209845a058349c6a108c7fa04c5b580 in term 0.
I20260811 18:40:34.327922 2833 raft_consensus.cc:2454] T 332701e1cc194d02a070ca30af47d4b8 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7209845a058349c6a108c7fa04c5b580 in term 0.
I20260811 18:40:34.328385 2197 leader_election.cc:302] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7209845a058349c6a108c7fa04c5b580, e4133e575b754c4fb437792fca3afd09; no voters:
I20260811 18:40:34.328683 3096 raft_consensus.cc:2781] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:40:34.328816 3096 raft_consensus.cc:482] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:40:34.328895 3096 raft_consensus.cc:3037] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:34.330941 3096 raft_consensus.cc:504] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:34.331425 3096 leader_election.cc:288] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580 [CANDIDATE]: Term 1 election: Requested vote from peers a1e13f628490435d85fa802f60e2d0f7 (127.30.185.130:44815), e4133e575b754c4fb437792fca3afd09 (127.30.185.131:34685)
I20260811 18:40:34.332016 2587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "332701e1cc194d02a070ca30af47d4b8" candidate_uuid: "7209845a058349c6a108c7fa04c5b580" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1e13f628490435d85fa802f60e2d0f7"
I20260811 18:40:34.332086 2833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "332701e1cc194d02a070ca30af47d4b8" candidate_uuid: "7209845a058349c6a108c7fa04c5b580" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4133e575b754c4fb437792fca3afd09"
I20260811 18:40:34.332258 2587 raft_consensus.cc:3037] T 332701e1cc194d02a070ca30af47d4b8 P a1e13f628490435d85fa802f60e2d0f7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:34.332309 2833 raft_consensus.cc:3037] T 332701e1cc194d02a070ca30af47d4b8 P e4133e575b754c4fb437792fca3afd09 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:34.334484 2587 raft_consensus.cc:2454] T 332701e1cc194d02a070ca30af47d4b8 P a1e13f628490435d85fa802f60e2d0f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7209845a058349c6a108c7fa04c5b580 in term 1.
I20260811 18:40:34.334486 2833 raft_consensus.cc:2454] T 332701e1cc194d02a070ca30af47d4b8 P e4133e575b754c4fb437792fca3afd09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7209845a058349c6a108c7fa04c5b580 in term 1.
I20260811 18:40:34.334942 2197 leader_election.cc:302] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7209845a058349c6a108c7fa04c5b580, a1e13f628490435d85fa802f60e2d0f7; no voters:
I20260811 18:40:34.335255 3096 raft_consensus.cc:2781] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:40:34.335412 3096 raft_consensus.cc:686] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580 [term 1 LEADER]: Becoming Leader. State: Replica: 7209845a058349c6a108c7fa04c5b580, State: Running, Role: LEADER
I20260811 18:40:34.335714 3096 consensus_queue.cc:237] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } }
I20260811 18:40:34.338488 2040 catalog_manager.cc:5495] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7209845a058349c6a108c7fa04c5b580 (127.30.185.129). New cstate: current_term: 1 leader_uuid: "7209845a058349c6a108c7fa04c5b580" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7209845a058349c6a108c7fa04c5b580" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 45641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 } health_report { overall_health: UNKNOWN } } }
I20260811 18:40:34.368063 3167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:34.362745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53846 (local address 127.30.185.129:45641)
0811 18:40:34.363253 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:34.363258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:34.363278 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:40:34.363337 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:34.363341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:34.363406 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:34.363489 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:40:34.363885 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:34.364704 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:34.366184 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:34.366609 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:34.366714 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:34.367241 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:40:34.367377 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:40:34.367453 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:34.367607 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 18:40:34.367671 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":263,"threads_started":1}
I20260811 18:40:34.382321 3167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:34.377009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:51515 (local address 127.30.185.129:45641)
0811 18:40:34.377270 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:34.377279 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:34.377607 (+ 328us) server_negotiation.cc:407] Connection header received
0811 18:40:34.377817 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:34.377826 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:34.377880 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:34.377972 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:40:34.378408 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:34.379208 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:34.380619 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:34.381012 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:34.381156 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:34.381557 (+ 401us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:40:34.381661 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:40:34.381775 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:34.381905 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 18:40:34.381970 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 18:40:34.382390 3171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:34.376858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:45641 (local address 127.30.185.129:51515)
0811 18:40:34.377507 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:34.377522 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:34.377681 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:34.377983 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:34.377987 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:34.377997 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:40:34.378237 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:34.378246 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:34.379327 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:34.379335 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:34.380482 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:34.380494 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:34.381222 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:34.381229 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:34.381360 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:34.381393 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:40:34.381773 (+ 380us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:40:34.381784 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 18:40:34.381886 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 18:40:34.382012 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":387,"threads_started":1}
I20260811 18:40:34.391070 2833 raft_consensus.cc:1261] T 332701e1cc194d02a070ca30af47d4b8 P e4133e575b754c4fb437792fca3afd09 [term 1 FOLLOWER]: Refusing update from remote peer 7209845a058349c6a108c7fa04c5b580: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:40:34.391232 2587 raft_consensus.cc:1261] T 332701e1cc194d02a070ca30af47d4b8 P a1e13f628490435d85fa802f60e2d0f7 [term 1 FOLLOWER]: Refusing update from remote peer 7209845a058349c6a108c7fa04c5b580: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:40:34.391654 3155 consensus_queue.cc:1035] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4133e575b754c4fb437792fca3afd09" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:40:34.392093 3065 consensus_queue.cc:1035] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1e13f628490435d85fa802f60e2d0f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:40:34.419296 3067 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 18:40:36.014008 2040 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:40:36.014276 2040 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:40:36.020305 2040 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb7753544bb949f6b8798dcaa4467435: Sending DeleteTablet for 3 replicas of tablet 332701e1cc194d02a070ca30af47d4b8
I20260811 18:40:36.021214 2289 tablet_service.cc:1514] Processing DeleteTablet for tablet 332701e1cc194d02a070ca30af47d4b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:36 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:34644
I20260811 18:40:36.021633 2543 tablet_service.cc:1514] Processing DeleteTablet for tablet 332701e1cc194d02a070ca30af47d4b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:36 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:56268
I20260811 18:40:36.021806 2794 tablet_service.cc:1514] Processing DeleteTablet for tablet 332701e1cc194d02a070ca30af47d4b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:36 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:56058
I20260811 18:40:36.022396 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 2171
I20260811 18:40:36.038345 3215 tablet_replica.cc:330] T 332701e1cc194d02a070ca30af47d4b8 P a1e13f628490435d85fa802f60e2d0f7: stopping tablet replica
I20260811 18:40:36.038606 3215 raft_consensus.cc:2229] T 332701e1cc194d02a070ca30af47d4b8 P a1e13f628490435d85fa802f60e2d0f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:40:36.038919 3215 raft_consensus.cc:2258] T 332701e1cc194d02a070ca30af47d4b8 P a1e13f628490435d85fa802f60e2d0f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:40:36.040143 3215 ts_tablet_manager.cc:1905] T 332701e1cc194d02a070ca30af47d4b8 P a1e13f628490435d85fa802f60e2d0f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:40:36.047477 3215 ts_tablet_manager.cc:1918] T 332701e1cc194d02a070ca30af47d4b8 P a1e13f628490435d85fa802f60e2d0f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:40:36.047657 3215 log.cc:1196] T 332701e1cc194d02a070ca30af47d4b8 P a1e13f628490435d85fa802f60e2d0f7: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/332701e1cc194d02a070ca30af47d4b8
I20260811 18:40:36.048156 3215 ts_tablet_manager.cc:1939] T 332701e1cc194d02a070ca30af47d4b8 P a1e13f628490435d85fa802f60e2d0f7: Deleting consensus metadata
Aug 11 18:40:36 dist-test-slave-z8x9 krb5kdc[1968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473633, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.185.129@KRBTEST.COM
I20260811 18:40:36.048889 3213 tablet_replica.cc:330] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580: stopping tablet replica
I20260811 18:40:36.049275 3213 raft_consensus.cc:2229] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:40:36.049569 2015 catalog_manager.cc:4841] TS a1e13f628490435d85fa802f60e2d0f7 (127.30.185.130:44815): tablet 332701e1cc194d02a070ca30af47d4b8 (table test-table [id=4d5868b58aed4cb8923b3a1bf48fae37]) successfully deleted
I20260811 18:40:36.050230 3213 raft_consensus.cc:2258] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:40:36.051594 3213 ts_tablet_manager.cc:1905] T 332701e1cc194d02a070ca30af47d4b8 P 7209845a058349c6a108c7fa04c5b580: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:40:36.057653 3216 tablet_replica.cc:330] T 332701e1cc194d02a070ca30af47d4b8 P e4133e575b754c4fb437792fca3afd09: stopping tablet replica
I20260811 18:40:36.057906 3216 raft_consensus.cc:2229] T 332701e1cc194d02a070ca30af47d4b8 P e4133e575b754c4fb437792fca3afd09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:40:36.058051 3220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:36.024565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53852 (local address 127.30.185.129:45641)
0811 18:40:36.037300 (+ 12735us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:36.037309 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:36.037343 (+ 34us) server_negotiation.cc:407] Connection header received
0811 18:40:36.037444 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:36.037452 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:36.037562 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:36.037708 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:36.039519 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:36.041235 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:36.043062 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:36.043755 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:36.043886 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:36.049852 (+ 5966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:36.049871 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:36.049894 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:36.049942 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:36.054445 (+ 4503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:36.055336 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:36.055345 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:36.055349 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:36.055563 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:36.056005 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:36.056010 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:36.056014 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:36.056460 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:36.056745 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:36.057390 (+ 645us) server_negotiation.cc:299] Negotiation successful
0811 18:40:36.057620 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12492,"thread_start_us":13294,"threads_started":1}
I20260811 18:40:36.058209 3216 raft_consensus.cc:2258] T 332701e1cc194d02a070ca30af47d4b8 P e4133e575b754c4fb437792fca3afd09 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:40:36.059674 2408 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53852
W20260811 18:40:36.060556 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 2171: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:40:36.060694 3216 ts_tablet_manager.cc:1905] T 332701e1cc194d02a070ca30af47d4b8 P e4133e575b754c4fb437792fca3afd09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:40:36.060768 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 2171
I20260811 18:40:36.083958 3216 ts_tablet_manager.cc:1918] T 332701e1cc194d02a070ca30af47d4b8 P e4133e575b754c4fb437792fca3afd09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:40:36.084147 3216 log.cc:1196] T 332701e1cc194d02a070ca30af47d4b8 P e4133e575b754c4fb437792fca3afd09: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/332701e1cc194d02a070ca30af47d4b8
I20260811 18:40:36.084648 3216 ts_tablet_manager.cc:1939] T 332701e1cc194d02a070ca30af47d4b8 P e4133e575b754c4fb437792fca3afd09: Deleting consensus metadata
I20260811 18:40:36.086164 2015 catalog_manager.cc:4841] TS e4133e575b754c4fb437792fca3afd09 (127.30.185.131:34685): tablet 332701e1cc194d02a070ca30af47d4b8 (table test-table [id=4d5868b58aed4cb8923b3a1bf48fae37]) successfully deleted
W20260811 18:40:36.107887 2018 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.185.129:45641 (error 108)
I20260811 18:40:36.108217 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 2424
I20260811 18:40:36.110558 3225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:36.109589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:45641 (local address 127.0.0.1:53860)
0811 18:40:36.110087 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:36.110299 (+ 212us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":246,"threads_started":1}
W20260811 18:40:36.110850 2018 catalog_manager.cc:4584] TS 7209845a058349c6a108c7fa04c5b580 (127.30.185.129:45641): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 332701e1cc194d02a070ca30af47d4b8: Network error: Client connection negotiation failed: client connection to 127.30.185.129:45641: connect: Connection refused (error 111)
Aug 11 18:40:36 dist-test-slave-z8x9 krb5kdc[1968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473633, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.185.130@KRBTEST.COM
I20260811 18:40:36.125351 3223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:36.109118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60156 (local address 127.30.185.130:44815)
0811 18:40:36.109878 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:36.109885 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:36.109905 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:40:36.109992 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:36.109997 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:36.110079 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:36.110171 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:36.111747 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:36.112538 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:36.113913 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:36.114295 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:36.114390 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:36.119032 (+ 4642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:36.119049 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:36.119066 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:36.119111 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:36.122265 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:36.123107 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:36.123115 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:36.123117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:36.123256 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:36.123640 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:36.123643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:36.123645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:36.124097 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:36.124330 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:36.124670 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 18:40:36.124871 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":387,"threads_started":1}
W20260811 18:40:36.126071 2654 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60156
W20260811 18:40:36.126768 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 2424: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:40:36.126968 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 2424
I20260811 18:40:36.163429 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 2666
I20260811 18:40:36.165395 3225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:36.164877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:45641 (local address 127.0.0.1:53872)
0811 18:40:36.165084 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:36.165221 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 18:40:36.166509 3225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:36.166064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:45641 (local address 127.0.0.1:53876)
0811 18:40:36.166234 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:36.166334 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
Aug 11 18:40:36 dist-test-slave-z8x9 krb5kdc[1968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473633, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.185.131@KRBTEST.COM
I20260811 18:40:36.180403 3227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:36.164234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32844 (local address 127.30.185.131:34685)
0811 18:40:36.164835 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:36.164840 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:36.164855 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:40:36.164906 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:36.164910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:36.165025 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:36.165160 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:36.167453 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:36.168211 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:36.169681 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:36.170168 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:36.170332 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:36.174764 (+ 4432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:36.174782 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:36.174798 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:36.174855 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:36.177844 (+ 2989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:36.178549 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:36.178557 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:36.178559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:36.178715 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:36.179074 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:36.179077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:36.179079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:36.179457 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:36.179690 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:36.179969 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 18:40:36.180149 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":737,"threads_started":1}
W20260811 18:40:36.181097 2912 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32844
W20260811 18:40:36.181885 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 2666: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:40:36.182056 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 2666
I20260811 18:40:36.211961 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 1992
I20260811 18:40:36.213274 2126 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:36.332857 3225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:36.332395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:45641 (local address 127.0.0.1:53882)
0811 18:40:36.332599 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:36.332697 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:45641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 18:40:36.333865 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 1992
2026-08-11T18:40:36Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9099 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:40:36 dist-test-slave-z8x9 krb5kdc[3232](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:40:36 dist-test-slave-z8x9 krb5kdc[3232](info): set up 2 sockets
Aug 11 18:40:36 dist-test-slave-z8x9 krb5kdc[3232](info): commencing operation
krb5kdc: starting...
W20260811 18:40:37.732692 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.341s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:40:37 dist-test-slave-z8x9 krb5kdc[3232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:40:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:40:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:40:37.988978 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:35481
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:45321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:35481
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:38.339064 3253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:38.339371 3253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:38.339460 3253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:38.346809 3253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:40:38.346925 3253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:38.346971 3253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:38.347031 3253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:40:38.347076 3253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:40:38.357029 3253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:45321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:35481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:35481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:38.358497 3253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:38.359760 3253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:38.372150 3273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:38.372314 3276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:38.372344 3274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:38.373633 3253 server_base.cc:1034] running on GCE node
I20260811 18:40:38.374415 3253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:38.375787 3253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:38.376999 3253 hybrid_clock.cc:648] HybridClock initialized: now 1786473638376954 us; error 36 us; skew 500 ppm
I20260811 18:40:38.377430 3253 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:38 dist-test-slave-z8x9 krb5kdc[3232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473638, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:38.380658 3253 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:40:38.382316 3253 webserver.cc:460] Webserver started at http://127.30.185.190:35537/ using document root <none> and password file <none>
I20260811 18:40:38.383057 3253 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:38.383214 3253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:38.383554 3253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:38.386641 3253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "425b0db22a104a359ad5161c614dcdf7"
format_stamp: "Formatted at 2026-08-11 18:40:38 on dist-test-slave-z8x9"
server_key: "f7104ac4f3f3ecf70b87ead2c66f6869"
server_key_iv: "9e433b0d6a7aa8607cee98c449b8e40c"
server_key_version: "encryptionkey@0"
I20260811 18:40:38.387427 3253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "425b0db22a104a359ad5161c614dcdf7"
format_stamp: "Formatted at 2026-08-11 18:40:38 on dist-test-slave-z8x9"
server_key: "f7104ac4f3f3ecf70b87ead2c66f6869"
server_key_iv: "9e433b0d6a7aa8607cee98c449b8e40c"
server_key_version: "encryptionkey@0"
I20260811 18:40:38.392781 3253 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:40:38.396361 3291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:38.397843 3253 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:40:38.398042 3253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "425b0db22a104a359ad5161c614dcdf7"
format_stamp: "Formatted at 2026-08-11 18:40:38 on dist-test-slave-z8x9"
server_key: "f7104ac4f3f3ecf70b87ead2c66f6869"
server_key_iv: "9e433b0d6a7aa8607cee98c449b8e40c"
server_key_version: "encryptionkey@0"
I20260811 18:40:38.398264 3253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:38.418442 3253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:38.422802 3253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:38.423115 3253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:38.443089 3253 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:35481
I20260811 18:40:38.443121 3378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:35481 every 8 connection(s)
I20260811 18:40:38.444667 3253 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:40:38.448585 3379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:38.454744 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 3253
I20260811 18:40:38.455003 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:40:38.455665 31462 external_mini_cluster.cc:1428] Setting key dd3a60eed9d9c6dd21adc0f8ec454243
I20260811 18:40:38.460345 3379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 425b0db22a104a359ad5161c614dcdf7: Bootstrap starting.
I20260811 18:40:38.464363 3379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 425b0db22a104a359ad5161c614dcdf7: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:38.465718 3379 log.cc:824] T 00000000000000000000000000000000 P 425b0db22a104a359ad5161c614dcdf7: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:38.469430 3379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 425b0db22a104a359ad5161c614dcdf7: No bootstrap required, opened a new log
Aug 11 18:40:38 dist-test-slave-z8x9 krb5kdc[3232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:38.478680 3379 raft_consensus.cc:348] T 00000000000000000000000000000000 P 425b0db22a104a359ad5161c614dcdf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "425b0db22a104a359ad5161c614dcdf7" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35481 } }
I20260811 18:40:38.479301 3379 raft_consensus.cc:374] T 00000000000000000000000000000000 P 425b0db22a104a359ad5161c614dcdf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:38.479454 3379 raft_consensus.cc:729] T 00000000000000000000000000000000 P 425b0db22a104a359ad5161c614dcdf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 425b0db22a104a359ad5161c614dcdf7, State: Initialized, Role: FOLLOWER
I20260811 18:40:38.480243 3379 consensus_queue.cc:260] T 00000000000000000000000000000000 P 425b0db22a104a359ad5161c614dcdf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "425b0db22a104a359ad5161c614dcdf7" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35481 } }
I20260811 18:40:38.480530 3379 raft_consensus.cc:388] T 00000000000000000000000000000000 P 425b0db22a104a359ad5161c614dcdf7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:40:38.480681 3379 raft_consensus.cc:482] T 00000000000000000000000000000000 P 425b0db22a104a359ad5161c614dcdf7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:40:38.480921 3379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 425b0db22a104a359ad5161c614dcdf7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:38.482167 3385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:38.457959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48830 (local address 127.30.185.190:35481)
0811 18:40:38.458810 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:38.458827 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:38.458883 (+ 56us) server_negotiation.cc:407] Connection header received
0811 18:40:38.461413 (+ 2530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:38.461442 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:38.461935 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:38.462449 (+ 514us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:38.463836 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:38.465940 (+ 2104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:38.467673 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:38.468754 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:38.468941 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:38.473571 (+ 4630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:38.473579 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:38.473600 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:38.473647 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:38.477101 (+ 3454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:38.477968 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:38.477984 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:38.477991 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:38.478167 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:38.478572 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:38.478577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:38.478580 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:38.479610 (+ 1030us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:38.479976 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:38.480438 (+ 462us) server_negotiation.cc:299] Negotiation successful
0811 18:40:38.480944 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":363,"threads_started":1}
I20260811 18:40:38.483556 3379 raft_consensus.cc:504] T 00000000000000000000000000000000 P 425b0db22a104a359ad5161c614dcdf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "425b0db22a104a359ad5161c614dcdf7" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35481 } }
I20260811 18:40:38.484098 3379 leader_election.cc:302] T 00000000000000000000000000000000 P 425b0db22a104a359ad5161c614dcdf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 425b0db22a104a359ad5161c614dcdf7; no voters:
I20260811 18:40:38.484829 3379 leader_election.cc:288] T 00000000000000000000000000000000 P 425b0db22a104a359ad5161c614dcdf7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:40:38.484980 3388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 425b0db22a104a359ad5161c614dcdf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:40:38.485745 3388 raft_consensus.cc:686] T 00000000000000000000000000000000 P 425b0db22a104a359ad5161c614dcdf7 [term 1 LEADER]: Becoming Leader. State: Replica: 425b0db22a104a359ad5161c614dcdf7, State: Running, Role: LEADER
I20260811 18:40:38.486362 3388 consensus_queue.cc:237] T 00000000000000000000000000000000 P 425b0db22a104a359ad5161c614dcdf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "425b0db22a104a359ad5161c614dcdf7" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35481 } }
I20260811 18:40:38.487250 3379 sys_catalog.cc:564] T 00000000000000000000000000000000 P 425b0db22a104a359ad5161c614dcdf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:40:38.489362 3390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 425b0db22a104a359ad5161c614dcdf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 425b0db22a104a359ad5161c614dcdf7. Latest consensus state: current_term: 1 leader_uuid: "425b0db22a104a359ad5161c614dcdf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "425b0db22a104a359ad5161c614dcdf7" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35481 } } }
I20260811 18:40:38.489742 3390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 425b0db22a104a359ad5161c614dcdf7 [sys.catalog]: This master's current role is: LEADER
I20260811 18:40:38.491987 3389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 425b0db22a104a359ad5161c614dcdf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "425b0db22a104a359ad5161c614dcdf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "425b0db22a104a359ad5161c614dcdf7" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35481 } } }
I20260811 18:40:38.492375 3389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 425b0db22a104a359ad5161c614dcdf7 [sys.catalog]: This master's current role is: LEADER
I20260811 18:40:38.493008 3401 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:40:38.498586 3401 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:40:38.508810 3401 catalog_manager.cc:1347] Generated new cluster ID: 8becd46c997342218b80b2d99e4f8a17
I20260811 18:40:38.508965 3401 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:40:38.523789 3401 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:40:38.525149 3401 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:40:38.547257 3401 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 425b0db22a104a359ad5161c614dcdf7: Generated new TSK 0
I20260811 18:40:38.548478 3401 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.185.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
I20260811 18:40:38.612519 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:35481
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:45321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:38.962625 3425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:38.962941 3425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:38.963119 3425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:38.970839 3425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:38.970956 3425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:38.971168 3425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:40:38.981407 3425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:45321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:35481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:38.982785 3425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:38.984146 3425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:38.998358 3435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:38.998423 3438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:38.998459 3433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:38.998870 3425 server_base.cc:1034] running on GCE node
I20260811 18:40:38.999650 3425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:39.000612 3425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:39.001858 3425 hybrid_clock.cc:648] HybridClock initialized: now 1786473639001794 us; error 50 us; skew 500 ppm
I20260811 18:40:39.002209 3425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:39 dist-test-slave-z8x9 krb5kdc[3232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473639, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:39.005662 3425 init.cc:377] Logged in from keytab as kudu/127.30.185.129@KRBTEST.COM (short username kudu)
I20260811 18:40:39.007267 3425 webserver.cc:460] Webserver started at http://127.30.185.129:40893/ using document root <none> and password file <none>
I20260811 18:40:39.007998 3425 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:39.008152 3425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:39.008508 3425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:39.011586 3425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "f1654fa4ad254f3e984c7e2285efc241"
format_stamp: "Formatted at 2026-08-11 18:40:39 on dist-test-slave-z8x9"
server_key: "c0210d9fbcf49a2e5ada4cc066c0ec73"
server_key_iv: "23887b75c7f51674c20155f4942df496"
server_key_version: "encryptionkey@0"
I20260811 18:40:39.012390 3425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "f1654fa4ad254f3e984c7e2285efc241"
format_stamp: "Formatted at 2026-08-11 18:40:39 on dist-test-slave-z8x9"
server_key: "c0210d9fbcf49a2e5ada4cc066c0ec73"
server_key_iv: "23887b75c7f51674c20155f4942df496"
server_key_version: "encryptionkey@0"
I20260811 18:40:39.017920 3425 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 18:40:39.021440 3450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:39.022814 3425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:40:39.023003 3425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "f1654fa4ad254f3e984c7e2285efc241"
format_stamp: "Formatted at 2026-08-11 18:40:39 on dist-test-slave-z8x9"
server_key: "c0210d9fbcf49a2e5ada4cc066c0ec73"
server_key_iv: "23887b75c7f51674c20155f4942df496"
server_key_version: "encryptionkey@0"
I20260811 18:40:39.023217 3425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:39.048802 3425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:39.053524 3425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:39.053833 3425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:39.054859 3425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:40:39.056212 3425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:39.056330 3425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:39.056481 3425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:39.056556 3425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:39.091289 3425 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:41579
I20260811 18:40:39.091327 3658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:41579 every 8 connection(s)
I20260811 18:40:39.092654 3425 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:40:39.098153 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 3425
I20260811 18:40:39.098379 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:40:39.099084 31462 external_mini_cluster.cc:1428] Setting key ea0b27b596deb00470f066ea4ceac659
Aug 11 18:40:39 dist-test-slave-z8x9 krb5kdc[3232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473639, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:39.117805 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:39.095547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:44741 (local address 127.30.185.190:35481)
0811 18:40:39.096025 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:39.096032 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:39.097000 (+ 968us) server_negotiation.cc:407] Connection header received
0811 18:40:39.099043 (+ 2043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:39.099051 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:39.099153 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:39.099302 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:39.101042 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:39.101850 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:39.104515 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:39.105237 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:39.105396 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:39.109924 (+ 4528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:39.109950 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:39.109957 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:39.110011 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:39.113881 (+ 3870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:39.114820 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:39.114829 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:39.114836 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:39.115000 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:39.115507 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:39.115515 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:39.115521 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:39.115941 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:39.116092 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:39.116966 (+ 874us) server_negotiation.cc:299] Negotiation successful
0811 18:40:39.117311 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":226,"threads_started":1}
I20260811 18:40:39.118542 3666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:39.096129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:35481 (local address 127.30.185.129:44741)
0811 18:40:39.096833 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:39.096874 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:39.098764 (+ 1890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:39.099504 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:39.099513 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:39.100035 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:39.100852 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:39.100887 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:39.102063 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:39.102070 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:39.104325 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:39.104344 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:39.105586 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:39.105595 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:39.106081 (+ 486us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:39.106925 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:39.106949 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:39.109659 (+ 2710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:39.114121 (+ 4462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:39.114130 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:39.114144 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:39.114672 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:39.115132 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:39.115136 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:39.115138 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:39.115383 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:39.116112 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:39.116118 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:39.116509 (+ 391us) client_negotiation.cc:769] Sending connection context
0811 18:40:39.116791 (+ 282us) client_negotiation.cc:240] Negotiation successful
0811 18:40:39.117324 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":272,"threads_started":1}
I20260811 18:40:39.120409 3660 heartbeater.cc:347] Connected to a master server at 127.30.185.190:35481
I20260811 18:40:39.120800 3660 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:39.121845 3660 heartbeater.cc:511] Master 127.30.185.190:35481 requested a full tablet report, sending...
I20260811 18:40:39.124542 3318 ts_manager.cc:194] Registered new tserver with Master: f1654fa4ad254f3e984c7e2285efc241 (127.30.185.129:41579)
WARNING: no policy specified for kudu/127.30.185.130@KRBTEST.COM; defaulting to no policy
I20260811 18:40:39.126163 3318 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:44741
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
I20260811 18:40:39.157016 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:35481
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:45321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:39.495551 3674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:39.495930 3674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:39.496117 3674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:39.503909 3674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:39.504076 3674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:39.504271 3674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:40:39.514575 3674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:45321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:35481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:39.516032 3674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:39.517493 3674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:39.531963 3682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:39.532001 3681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:39.532325 3674 server_base.cc:1034] running on GCE node
W20260811 18:40:39.532050 3686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:39.533464 3674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:39.534431 3674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:39.535729 3674 hybrid_clock.cc:648] HybridClock initialized: now 1786473639535673 us; error 43 us; skew 500 ppm
I20260811 18:40:39.536155 3674 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:39 dist-test-slave-z8x9 krb5kdc[3232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473639, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:39.540071 3674 init.cc:377] Logged in from keytab as kudu/127.30.185.130@KRBTEST.COM (short username kudu)
I20260811 18:40:39.541934 3674 webserver.cc:460] Webserver started at http://127.30.185.130:37321/ using document root <none> and password file <none>
I20260811 18:40:39.542747 3674 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:39.542914 3674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:39.543282 3674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:39.546663 3674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "faad9e87ab274e06b189580337b2f9fe"
format_stamp: "Formatted at 2026-08-11 18:40:39 on dist-test-slave-z8x9"
server_key: "9049b60889862c24cb9c966ab7af68dc"
server_key_iv: "c759b7ee20a57fac68b50a72bdc6b87c"
server_key_version: "encryptionkey@0"
I20260811 18:40:39.547572 3674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "faad9e87ab274e06b189580337b2f9fe"
format_stamp: "Formatted at 2026-08-11 18:40:39 on dist-test-slave-z8x9"
server_key: "9049b60889862c24cb9c966ab7af68dc"
server_key_iv: "c759b7ee20a57fac68b50a72bdc6b87c"
server_key_version: "encryptionkey@0"
I20260811 18:40:39.553278 3674 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 18:40:39.556833 3698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:39.558307 3674 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:40:39.558498 3674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "faad9e87ab274e06b189580337b2f9fe"
format_stamp: "Formatted at 2026-08-11 18:40:39 on dist-test-slave-z8x9"
server_key: "9049b60889862c24cb9c966ab7af68dc"
server_key_iv: "c759b7ee20a57fac68b50a72bdc6b87c"
server_key_version: "encryptionkey@0"
I20260811 18:40:39.558707 3674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:39.575811 3674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:39.580495 3674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:39.580813 3674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:39.581835 3674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:40:39.583278 3674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:39.583400 3674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:39.583554 3674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:39.583627 3674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:39.618062 3674 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:38631
I20260811 18:40:39.618201 3886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:38631 every 8 connection(s)
I20260811 18:40:39.619573 3674 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:40:39.622517 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 3674
I20260811 18:40:39.622722 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:40:39.623273 31462 external_mini_cluster.cc:1428] Setting key ba639c22a3ac060ee1b6bc409d8542f6
Aug 11 18:40:39 dist-test-slave-z8x9 krb5kdc[3232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473639, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:39.642619 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:39.622252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:59349 (local address 127.30.185.190:35481)
0811 18:40:39.622789 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:39.622798 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:39.623864 (+ 1066us) server_negotiation.cc:407] Connection header received
0811 18:40:39.626052 (+ 2188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:39.626059 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:39.626146 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:39.626294 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:39.628039 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:39.629378 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:39.630989 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:39.631836 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:39.631998 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:39.636041 (+ 4043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:39.636056 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:39.636060 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:39.636096 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:39.639110 (+ 3014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:39.639945 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:39.639958 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:39.639962 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:39.640104 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:39.640593 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:39.640600 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:39.640604 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:39.640917 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:39.641102 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:39.641943 (+ 841us) server_negotiation.cc:299] Negotiation successful
0811 18:40:39.642196 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":271,"threads_started":1}
I20260811 18:40:39.643769 3892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:39.622774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:35481 (local address 127.30.185.130:59349)
0811 18:40:39.623678 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:39.623733 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:39.625758 (+ 2025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:39.626541 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:39.626549 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:39.627157 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:39.627810 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:39.627829 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:39.629544 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:39.629548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:39.630827 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:39.630839 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:39.632058 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:39.632064 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:39.632268 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:39.633059 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:39.633080 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:39.635779 (+ 2699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:39.639280 (+ 3501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:39.639290 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:39.639308 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:39.639800 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:39.640239 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:39.640244 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:39.640248 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:39.640466 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:39.641066 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:39.641075 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:39.641549 (+ 474us) client_negotiation.cc:769] Sending connection context
0811 18:40:39.641858 (+ 309us) client_negotiation.cc:240] Negotiation successful
0811 18:40:39.642463 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":356,"threads_started":1}
I20260811 18:40:39.645524 3887 heartbeater.cc:347] Connected to a master server at 127.30.185.190:35481
I20260811 18:40:39.645895 3887 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:39.646881 3887 heartbeater.cc:511] Master 127.30.185.190:35481 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.185.131@KRBTEST.COM; defaulting to no policy
I20260811 18:40:39.648693 3318 ts_manager.cc:194] Registered new tserver with Master: faad9e87ab274e06b189580337b2f9fe (127.30.185.130:38631)
I20260811 18:40:39.649593 3318 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:59349
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
I20260811 18:40:39.680972 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:35481
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:45321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:40.023859 3896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:40.024196 3896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:40.024379 3896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:40.032398 3896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:40.032558 3896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:40.032749 3896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:40:40.042904 3896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:45321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:35481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:40.044394 3896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:40.045807 3896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:40.059293 3910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:40.059330 3908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:40.059393 3913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:40.060424 3896 server_base.cc:1034] running on GCE node
I20260811 18:40:40.061087 3896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:40.062212 3896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:40.063474 3896 hybrid_clock.cc:648] HybridClock initialized: now 1786473640063417 us; error 30 us; skew 500 ppm
I20260811 18:40:40.063814 3896 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:40 dist-test-slave-z8x9 krb5kdc[3232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473640, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:40.067225 3896 init.cc:377] Logged in from keytab as kudu/127.30.185.131@KRBTEST.COM (short username kudu)
I20260811 18:40:40.068759 3896 webserver.cc:460] Webserver started at http://127.30.185.131:46507/ using document root <none> and password file <none>
I20260811 18:40:40.069572 3896 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:40.069725 3896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:40.070051 3896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:40.073019 3896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "6cb19c9fa7924104b824a2105ff180c7"
format_stamp: "Formatted at 2026-08-11 18:40:40 on dist-test-slave-z8x9"
server_key: "ddd9288492ba0d91d9c8d5073966b9a5"
server_key_iv: "af7ffbcdfc7d2d1af013b0689e0eeb47"
server_key_version: "encryptionkey@0"
I20260811 18:40:40.073935 3896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "6cb19c9fa7924104b824a2105ff180c7"
format_stamp: "Formatted at 2026-08-11 18:40:40 on dist-test-slave-z8x9"
server_key: "ddd9288492ba0d91d9c8d5073966b9a5"
server_key_iv: "af7ffbcdfc7d2d1af013b0689e0eeb47"
server_key_version: "encryptionkey@0"
I20260811 18:40:40.079381 3896 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:40:40.082998 3936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:40.084321 3896 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:40:40.084515 3896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "6cb19c9fa7924104b824a2105ff180c7"
format_stamp: "Formatted at 2026-08-11 18:40:40 on dist-test-slave-z8x9"
server_key: "ddd9288492ba0d91d9c8d5073966b9a5"
server_key_iv: "af7ffbcdfc7d2d1af013b0689e0eeb47"
server_key_version: "encryptionkey@0"
I20260811 18:40:40.084739 3896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:40.102566 3896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:40.107172 3896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:40.107574 3896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:40.108995 3896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:40:40.110934 3896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:40.111012 3896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:40.111110 3896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:40.111157 3896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:40.130272 3660 heartbeater.cc:503] Master 127.30.185.190:35481 was elected leader, sending a full tablet report...
I20260811 18:40:40.150141 3896 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:42129
I20260811 18:40:40.150188 4140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:42129 every 8 connection(s)
I20260811 18:40:40.151485 3896 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:40:40.156404 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 3896
I20260811 18:40:40.156605 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:40:40.157229 31462 external_mini_cluster.cc:1428] Setting key f7f302aeb89027bbf3e2ff2d134c938f
Aug 11 18:40:40 dist-test-slave-z8x9 krb5kdc[3232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473640, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:40.174458 4147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:40.154228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:37013 (local address 127.30.185.190:35481)
0811 18:40:40.154731 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:40.154736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:40.155599 (+ 863us) server_negotiation.cc:407] Connection header received
0811 18:40:40.158285 (+ 2686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:40.158291 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:40.158391 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:40.158519 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:40.160297 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:40.161302 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:40.163148 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:40.163591 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:40.163733 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:40.167557 (+ 3824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:40.167568 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:40.167571 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:40.167607 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:40.170658 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:40.171691 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:40.171699 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:40.171707 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:40.171850 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:40.172481 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:40.172488 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:40.172491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:40.172863 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:40.172991 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:40.173968 (+ 977us) server_negotiation.cc:299] Negotiation successful
0811 18:40:40.174158 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":263,"threads_started":1}
I20260811 18:40:40.175452 4150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:40.154649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:35481 (local address 127.30.185.131:37013)
0811 18:40:40.155409 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:40.155455 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:40.157980 (+ 2525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:40.158826 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:40.158838 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:40.159506 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:40.160109 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:40.160127 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:40.161485 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:40.161490 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:40.162977 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:40.162992 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:40.163860 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:40.163869 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:40.164109 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:40.164878 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:40.164903 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:40.167306 (+ 2403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:40.170889 (+ 3583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:40.170897 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:40.170913 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:40.171546 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:40.171991 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:40.171995 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:40.171998 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:40.172319 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:40.173006 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:40.173015 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:40.173587 (+ 572us) client_negotiation.cc:769] Sending connection context
0811 18:40:40.173852 (+ 265us) client_negotiation.cc:240] Negotiation successful
0811 18:40:40.174291 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":308,"threads_started":1}
I20260811 18:40:40.176862 4141 heartbeater.cc:347] Connected to a master server at 127.30.185.190:35481
I20260811 18:40:40.177263 4141 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:40.178022 4141 heartbeater.cc:511] Master 127.30.185.190:35481 requested a full tablet report, sending...
I20260811 18:40:40.179528 3318 ts_manager.cc:194] Registered new tserver with Master: 6cb19c9fa7924104b824a2105ff180c7 (127.30.185.131:42129)
I20260811 18:40:40.180408 3318 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:37013
I20260811 18:40:40.185963 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 18:40:40.229938 4147 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:40:40.227322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48834 (local address 127.30.185.190:35481)
0811 18:40:40.227505 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:40.227509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:40.227998 (+ 489us) server_negotiation.cc:407] Connection header received
0811 18:40:40.228208 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:40.228212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:40.228296 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:40.228402 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:40.229722 (+ 1320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48834: BlockingRecv error: recv got EOF from 127.0.0.1:48834 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 18:40:40.233357 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 3425
W20260811 18:40:40.236457 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 3425: Not authorized: Client connection negotiation failed: client connection to 127.30.185.129:41579: server requires authentication, but client does not have Kerberos credentials available
I20260811 18:40:40.236790 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 3425
I20260811 18:40:40.261794 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 3674
W20260811 18:40:40.264663 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 3674: Not authorized: Client connection negotiation failed: client connection to 127.30.185.130:38631: server requires authentication, but client does not have Kerberos credentials available
I20260811 18:40:40.264845 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 3674
I20260811 18:40:40.291258 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 3896
W20260811 18:40:40.294629 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 3896: Not authorized: Client connection negotiation failed: client connection to 127.30.185.131:42129: server requires authentication, but client does not have Kerberos credentials available
I20260811 18:40:40.294777 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 3896
I20260811 18:40:40.324563 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 3253
I20260811 18:40:40.326126 3369 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:40.433099 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 3253
2026-08-11T18:40:40Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4094 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:40:40 dist-test-slave-z8x9 krb5kdc[4184](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:40:40 dist-test-slave-z8x9 krb5kdc[4184](info): set up 2 sockets
Aug 11 18:40:40 dist-test-slave-z8x9 krb5kdc[4184](info): commencing operation
krb5kdc: starting...
W20260811 18:40:41.823941 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.338s user 0.007s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:40:41 dist-test-slave-z8x9 krb5kdc[4184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:40:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:40:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:40:42.085620 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:41261
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:45693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:41261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:42.425375 4216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:42.425678 4216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:42.425833 4216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:42.433315 4216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:40:42.433455 4216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:42.433565 4216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:42.433665 4216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:40:42.433758 4216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:40:42.443773 4216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:45693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:41261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:41261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:42.445123 4216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:42.446463 4216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:42.458950 4227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:42.458871 4232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:42.458913 4228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:42.459983 4216 server_base.cc:1034] running on GCE node
I20260811 18:40:42.460830 4216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:42.462340 4216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:42.463579 4216 hybrid_clock.cc:648] HybridClock initialized: now 1786473642463541 us; error 56 us; skew 500 ppm
I20260811 18:40:42.463958 4216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:42 dist-test-slave-z8x9 krb5kdc[4184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473642, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:42.471522 4216 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:40:42.473261 4216 webserver.cc:460] Webserver started at http://127.30.185.190:43865/ using document root <none> and password file <none>
I20260811 18:40:42.473986 4216 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:42.474136 4216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:42.474475 4216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:42.477663 4216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "08873900e6b04154951aba836874db7c"
format_stamp: "Formatted at 2026-08-11 18:40:42 on dist-test-slave-z8x9"
server_key: "a8a87d8d59343f61d18da0fc980ee455"
server_key_iv: "da744d45c4adf0e94928dd81cedad9c9"
server_key_version: "encryptionkey@0"
I20260811 18:40:42.478482 4216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "08873900e6b04154951aba836874db7c"
format_stamp: "Formatted at 2026-08-11 18:40:42 on dist-test-slave-z8x9"
server_key: "a8a87d8d59343f61d18da0fc980ee455"
server_key_iv: "da744d45c4adf0e94928dd81cedad9c9"
server_key_version: "encryptionkey@0"
I20260811 18:40:42.483933 4216 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:40:42.487473 4244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:42.488768 4216 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 18:40:42.488953 4216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "08873900e6b04154951aba836874db7c"
format_stamp: "Formatted at 2026-08-11 18:40:42 on dist-test-slave-z8x9"
server_key: "a8a87d8d59343f61d18da0fc980ee455"
server_key_iv: "da744d45c4adf0e94928dd81cedad9c9"
server_key_version: "encryptionkey@0"
I20260811 18:40:42.489156 4216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:42.507398 4216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:42.511968 4216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:42.512272 4216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:42.532217 4216 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:41261
I20260811 18:40:42.532274 4338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:41261 every 8 connection(s)
I20260811 18:40:42.533706 4216 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:40:42.537915 4339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:42.541450 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 4216
I20260811 18:40:42.541695 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:40:42.542356 31462 external_mini_cluster.cc:1428] Setting key 828257a7731e154bfba78ad6b224ce7f
I20260811 18:40:42.550726 4339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08873900e6b04154951aba836874db7c: Bootstrap starting.
I20260811 18:40:42.554024 4339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08873900e6b04154951aba836874db7c: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:42.555068 4339 log.cc:824] T 00000000000000000000000000000000 P 08873900e6b04154951aba836874db7c: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:40:42 dist-test-slave-z8x9 krb5kdc[4184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:42.558315 4339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08873900e6b04154951aba836874db7c: No bootstrap required, opened a new log
I20260811 18:40:42.566138 4344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:42.544387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34910 (local address 127.30.185.190:41261)
0811 18:40:42.545369 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:42.545381 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:42.545423 (+ 42us) server_negotiation.cc:407] Connection header received
0811 18:40:42.547141 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:42.547165 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:42.547595 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:42.548015 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:42.549694 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:42.551050 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:42.552387 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:42.553083 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:42.553336 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:42.558047 (+ 4711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:42.558055 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:42.558070 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:42.558106 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:42.561432 (+ 3326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:42.562177 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:42.562190 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:42.562197 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:42.562417 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:42.562836 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:42.562841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:42.562855 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:42.563482 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:42.563851 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:42.564307 (+ 456us) server_negotiation.cc:299] Negotiation successful
0811 18:40:42.564793 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":431,"threads_started":1}
I20260811 18:40:42.566541 4339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08873900e6b04154951aba836874db7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08873900e6b04154951aba836874db7c" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41261 } }
I20260811 18:40:42.566941 4339 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08873900e6b04154951aba836874db7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:42.567065 4339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08873900e6b04154951aba836874db7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08873900e6b04154951aba836874db7c, State: Initialized, Role: FOLLOWER
I20260811 18:40:42.567653 4339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08873900e6b04154951aba836874db7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08873900e6b04154951aba836874db7c" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41261 } }
I20260811 18:40:42.567924 4339 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08873900e6b04154951aba836874db7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:40:42.568086 4339 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08873900e6b04154951aba836874db7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:40:42.568269 4339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08873900e6b04154951aba836874db7c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:42.570587 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:40:42.571116 4339 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08873900e6b04154951aba836874db7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08873900e6b04154951aba836874db7c" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41261 } }
I20260811 18:40:42.571663 4339 leader_election.cc:302] T 00000000000000000000000000000000 P 08873900e6b04154951aba836874db7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08873900e6b04154951aba836874db7c; no voters:
I20260811 18:40:42.572383 4339 leader_election.cc:288] T 00000000000000000000000000000000 P 08873900e6b04154951aba836874db7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:40:42.572489 4347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08873900e6b04154951aba836874db7c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:40:42.575246 4339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08873900e6b04154951aba836874db7c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:40:42.579524 4347 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08873900e6b04154951aba836874db7c [term 1 LEADER]: Becoming Leader. State: Replica: 08873900e6b04154951aba836874db7c, State: Running, Role: LEADER
I20260811 18:40:42.580480 4347 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08873900e6b04154951aba836874db7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08873900e6b04154951aba836874db7c" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41261 } }
I20260811 18:40:42.586095 4349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08873900e6b04154951aba836874db7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08873900e6b04154951aba836874db7c. Latest consensus state: current_term: 1 leader_uuid: "08873900e6b04154951aba836874db7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08873900e6b04154951aba836874db7c" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41261 } } }
I20260811 18:40:42.586635 4349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08873900e6b04154951aba836874db7c [sys.catalog]: This master's current role is: LEADER
I20260811 18:40:42.587361 4348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08873900e6b04154951aba836874db7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08873900e6b04154951aba836874db7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08873900e6b04154951aba836874db7c" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41261 } } }
I20260811 18:40:42.587694 4348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08873900e6b04154951aba836874db7c [sys.catalog]: This master's current role is: LEADER
I20260811 18:40:42.590283 4369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:40:42.596459 4369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:40:42.611443 4386 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 18:40:42.614877 4369 catalog_manager.cc:1347] Generated new cluster ID: 55f7acf207d34aaa84598ef4311877d2
I20260811 18:40:42.615015 4369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:40:42.632869 4369 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:40:42.633431 4344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:42.624775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34914 (local address 127.30.185.190:41261)
0811 18:40:42.624997 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:42.625007 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:42.625690 (+ 683us) server_negotiation.cc:407] Connection header received
0811 18:40:42.625928 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:42.625933 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:42.626010 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:42.626131 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:42.627242 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:42.628039 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:42.629462 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:42.629917 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:42.630080 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:42.631413 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:42.631418 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:42.631422 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:40:42.632407 (+ 985us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:42.632500 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:40:42.632541 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:42.632658 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:42.632894 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 18:40:42.633002 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":72}
I20260811 18:40:42.634879 4369 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:40:42.645589 4369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08873900e6b04154951aba836874db7c: Generated new TSK 0
I20260811 18:40:42.646497 4369 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:40:42.656535 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 4216
I20260811 18:40:42.658417 4329 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:42.757999 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 4216
2026-08-11T18:40:42Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2320 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:40:42 dist-test-slave-z8x9 krb5kdc[4403](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:40:42 dist-test-slave-z8x9 krb5kdc[4403](info): set up 2 sockets
Aug 11 18:40:42 dist-test-slave-z8x9 krb5kdc[4403](info): commencing operation
krb5kdc: starting...
W20260811 18:40:44.173797 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.368s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:40:44 dist-test-slave-z8x9 krb5kdc[4403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:40:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:40:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:40:44.427145 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:41299
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:42975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:41299
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:44.767228 4425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:44.767557 4425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:44.767751 4425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:44.775110 4425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:40:44.775243 4425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:44.775297 4425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:44.775405 4425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:40:44.775499 4425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:40:44.785725 4425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:42975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:41299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:41299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:44.787207 4425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:44.788661 4425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:44.801340 4439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:44.801262 4438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:44.802001 4441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:44.802273 4425 server_base.cc:1034] running on GCE node
I20260811 18:40:44.803150 4425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:44.804692 4425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:44.805943 4425 hybrid_clock.cc:648] HybridClock initialized: now 1786473644805909 us; error 46 us; skew 500 ppm
I20260811 18:40:44.806309 4425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:44 dist-test-slave-z8x9 krb5kdc[4403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473644, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:44.809643 4425 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:40:44.811246 4425 webserver.cc:460] Webserver started at http://127.30.185.190:44455/ using document root <none> and password file <none>
I20260811 18:40:44.811975 4425 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:44.812134 4425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:44.812471 4425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:44.815873 4425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "c7d4fdf658d54c5e83d9d7b5ddb9b39c"
format_stamp: "Formatted at 2026-08-11 18:40:44 on dist-test-slave-z8x9"
server_key: "0785b10805146fbc2e690721594975fa"
server_key_iv: "72f3b36dbcc5c14b52113270304761f3"
server_key_version: "encryptionkey@0"
I20260811 18:40:44.816797 4425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "c7d4fdf658d54c5e83d9d7b5ddb9b39c"
format_stamp: "Formatted at 2026-08-11 18:40:44 on dist-test-slave-z8x9"
server_key: "0785b10805146fbc2e690721594975fa"
server_key_iv: "72f3b36dbcc5c14b52113270304761f3"
server_key_version: "encryptionkey@0"
I20260811 18:40:44.823181 4425 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260811 18:40:44.827116 4450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:44.828608 4425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:40:44.828823 4425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "c7d4fdf658d54c5e83d9d7b5ddb9b39c"
format_stamp: "Formatted at 2026-08-11 18:40:44 on dist-test-slave-z8x9"
server_key: "0785b10805146fbc2e690721594975fa"
server_key_iv: "72f3b36dbcc5c14b52113270304761f3"
server_key_version: "encryptionkey@0"
I20260811 18:40:44.829064 4425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:44.854674 4425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:44.858994 4425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:44.859316 4425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:44.880164 4425 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:41299
I20260811 18:40:44.880200 4552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:41299 every 8 connection(s)
I20260811 18:40:44.881685 4425 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:40:44.883038 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 4425
I20260811 18:40:44.883276 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:40:44.884101 31462 external_mini_cluster.cc:1428] Setting key 2daf9b222f3e459604432d0b73635fd0
I20260811 18:40:44.886556 4559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:44.897572 4559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7d4fdf658d54c5e83d9d7b5ddb9b39c: Bootstrap starting.
Aug 11 18:40:44 dist-test-slave-z8x9 krb5kdc[4403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:44.900754 4559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7d4fdf658d54c5e83d9d7b5ddb9b39c: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:44.902202 4559 log.cc:824] T 00000000000000000000000000000000 P c7d4fdf658d54c5e83d9d7b5ddb9b39c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:44.905632 4559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7d4fdf658d54c5e83d9d7b5ddb9b39c: No bootstrap required, opened a new log
I20260811 18:40:44.907541 4566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:44.886380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46726 (local address 127.30.185.190:41299)
0811 18:40:44.887181 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:44.887193 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:44.887233 (+ 40us) server_negotiation.cc:407] Connection header received
0811 18:40:44.888892 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:44.888913 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:44.889365 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:44.889780 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:44.891245 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:44.892560 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:44.893980 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:44.894696 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:44.894924 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:44.899094 (+ 4170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:44.899102 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:44.899116 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:44.899152 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:44.902796 (+ 3644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:44.903533 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:44.903540 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:44.903546 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:44.903714 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:44.904175 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:44.904181 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:44.904184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:44.904924 (+ 740us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:44.905289 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:44.905750 (+ 461us) server_negotiation.cc:299] Negotiation successful
0811 18:40:44.906216 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":374,"threads_started":1}
I20260811 18:40:44.914433 4559 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7d4fdf658d54c5e83d9d7b5ddb9b39c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d4fdf658d54c5e83d9d7b5ddb9b39c" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41299 } }
I20260811 18:40:44.914764 4559 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7d4fdf658d54c5e83d9d7b5ddb9b39c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:44.914837 4559 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7d4fdf658d54c5e83d9d7b5ddb9b39c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7d4fdf658d54c5e83d9d7b5ddb9b39c, State: Initialized, Role: FOLLOWER
I20260811 18:40:44.915297 4559 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7d4fdf658d54c5e83d9d7b5ddb9b39c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d4fdf658d54c5e83d9d7b5ddb9b39c" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41299 } }
I20260811 18:40:44.915498 4559 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7d4fdf658d54c5e83d9d7b5ddb9b39c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:40:44.915592 4559 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7d4fdf658d54c5e83d9d7b5ddb9b39c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:40:44.915716 4559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7d4fdf658d54c5e83d9d7b5ddb9b39c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:44.918123 4559 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7d4fdf658d54c5e83d9d7b5ddb9b39c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d4fdf658d54c5e83d9d7b5ddb9b39c" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41299 } }
I20260811 18:40:44.918565 4559 leader_election.cc:302] T 00000000000000000000000000000000 P c7d4fdf658d54c5e83d9d7b5ddb9b39c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7d4fdf658d54c5e83d9d7b5ddb9b39c; no voters:
I20260811 18:40:44.919117 4559 leader_election.cc:288] T 00000000000000000000000000000000 P c7d4fdf658d54c5e83d9d7b5ddb9b39c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:40:44.919231 4568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7d4fdf658d54c5e83d9d7b5ddb9b39c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:40:44.919974 4568 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7d4fdf658d54c5e83d9d7b5ddb9b39c [term 1 LEADER]: Becoming Leader. State: Replica: c7d4fdf658d54c5e83d9d7b5ddb9b39c, State: Running, Role: LEADER
I20260811 18:40:44.920503 4568 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7d4fdf658d54c5e83d9d7b5ddb9b39c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d4fdf658d54c5e83d9d7b5ddb9b39c" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41299 } }
I20260811 18:40:44.921311 4559 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7d4fdf658d54c5e83d9d7b5ddb9b39c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:40:44.923583 4571 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7d4fdf658d54c5e83d9d7b5ddb9b39c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7d4fdf658d54c5e83d9d7b5ddb9b39c. Latest consensus state: current_term: 1 leader_uuid: "c7d4fdf658d54c5e83d9d7b5ddb9b39c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d4fdf658d54c5e83d9d7b5ddb9b39c" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41299 } } }
I20260811 18:40:44.923941 4571 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7d4fdf658d54c5e83d9d7b5ddb9b39c [sys.catalog]: This master's current role is: LEADER
I20260811 18:40:44.923687 4569 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7d4fdf658d54c5e83d9d7b5ddb9b39c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7d4fdf658d54c5e83d9d7b5ddb9b39c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d4fdf658d54c5e83d9d7b5ddb9b39c" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41299 } } }
I20260811 18:40:44.924140 4569 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7d4fdf658d54c5e83d9d7b5ddb9b39c [sys.catalog]: This master's current role is: LEADER
I20260811 18:40:44.932976 4576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:40:44.937665 4576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:40:44.945906 4576 catalog_manager.cc:1347] Generated new cluster ID: 12646353c36849dd955c7db02e8fec6a
I20260811 18:40:44.946033 4576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:40:44.955647 4576 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:40:44.957360 4576 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:40:44.965968 4576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7d4fdf658d54c5e83d9d7b5ddb9b39c: Generated new TSK 0
I20260811 18:40:44.966931 4576 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.185.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
I20260811 18:40:45.032932 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41299
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:42975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:45.377677 4607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:45.377970 4607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:45.378165 4607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:45.385600 4607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:45.385716 4607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:45.385932 4607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:40:45.395664 4607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:42975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41299
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:45.397037 4607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:45.398442 4607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:45.412269 4615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:45.412541 4617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:45.412297 4619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:45.413149 4607 server_base.cc:1034] running on GCE node
I20260811 18:40:45.413851 4607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:45.414835 4607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:45.416071 4607 hybrid_clock.cc:648] HybridClock initialized: now 1786473645416002 us; error 29 us; skew 500 ppm
I20260811 18:40:45.416447 4607 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:45 dist-test-slave-z8x9 krb5kdc[4403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473645, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:45.419895 4607 init.cc:377] Logged in from keytab as kudu/127.30.185.129@KRBTEST.COM (short username kudu)
I20260811 18:40:45.421527 4607 webserver.cc:460] Webserver started at http://127.30.185.129:43551/ using document root <none> and password file <none>
I20260811 18:40:45.422271 4607 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:45.422428 4607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:45.422747 4607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:45.425776 4607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "ff5cd29ceb0c45cea027db151fc77483"
format_stamp: "Formatted at 2026-08-11 18:40:45 on dist-test-slave-z8x9"
server_key: "852e4e38198347414874b2b377f9c585"
server_key_iv: "f7cd21d4f45879f70d248b07018a2392"
server_key_version: "encryptionkey@0"
I20260811 18:40:45.426644 4607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "ff5cd29ceb0c45cea027db151fc77483"
format_stamp: "Formatted at 2026-08-11 18:40:45 on dist-test-slave-z8x9"
server_key: "852e4e38198347414874b2b377f9c585"
server_key_iv: "f7cd21d4f45879f70d248b07018a2392"
server_key_version: "encryptionkey@0"
I20260811 18:40:45.432055 4607 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:40:45.435885 4634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:45.437450 4607 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 18:40:45.437640 4607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "ff5cd29ceb0c45cea027db151fc77483"
format_stamp: "Formatted at 2026-08-11 18:40:45 on dist-test-slave-z8x9"
server_key: "852e4e38198347414874b2b377f9c585"
server_key_iv: "f7cd21d4f45879f70d248b07018a2392"
server_key_version: "encryptionkey@0"
I20260811 18:40:45.437855 4607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:45.462606 4607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:45.467515 4607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:45.467849 4607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:45.469012 4607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:40:45.470456 4607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:45.470566 4607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:45.470715 4607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:45.470790 4607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:45.505577 4607 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:36761
I20260811 18:40:45.505637 4863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:36761 every 8 connection(s)
I20260811 18:40:45.506946 4607 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:40:45.508208 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 4607
I20260811 18:40:45.508450 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:40:45.509325 31462 external_mini_cluster.cc:1428] Setting key af04641233a96d6b625e98995dd3efaf
Aug 11 18:40:45 dist-test-slave-z8x9 krb5kdc[4403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473645, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:45.530555 4868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:45.509973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:33667 (local address 127.30.185.190:41299)
0811 18:40:45.510528 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:45.510534 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:45.511611 (+ 1077us) server_negotiation.cc:407] Connection header received
0811 18:40:45.513530 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:45.513540 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:45.513633 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:45.513765 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:45.515235 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:45.516426 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:45.518090 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:45.518786 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:45.518944 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:45.522923 (+ 3979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:45.522933 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:45.522941 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:45.522988 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:45.526803 (+ 3815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:45.527571 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:45.527579 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:45.527585 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:45.527742 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:45.528181 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:45.528188 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:45.528194 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:45.528657 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:45.528829 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:45.529736 (+ 907us) server_negotiation.cc:299] Negotiation successful
0811 18:40:45.530042 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":266,"threads_started":1}
I20260811 18:40:45.531263 4871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:45.510605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41299 (local address 127.30.185.129:33667)
0811 18:40:45.511445 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:45.511494 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:45.513257 (+ 1763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:45.513967 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:45.513976 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:45.514487 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:45.515051 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:45.515069 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:45.516592 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:45.516595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:45.517931 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:45.517944 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:45.519004 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:45.519008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:45.519209 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:45.519945 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:45.519967 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:45.522681 (+ 2714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:45.526970 (+ 4289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:45.526977 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:45.526990 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:45.527427 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:45.527861 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:45.527864 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:45.527866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:45.528058 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:45.528829 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:45.528836 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:45.529284 (+ 448us) client_negotiation.cc:769] Sending connection context
0811 18:40:45.529548 (+ 264us) client_negotiation.cc:240] Negotiation successful
0811 18:40:45.530057 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":359,"threads_started":1}
I20260811 18:40:45.533327 4865 heartbeater.cc:347] Connected to a master server at 127.30.185.190:41299
I20260811 18:40:45.533664 4865 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:45.534552 4865 heartbeater.cc:511] Master 127.30.185.190:41299 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.185.130@KRBTEST.COM; defaulting to no policy
I20260811 18:40:45.536880 4484 ts_manager.cc:194] Registered new tserver with Master: ff5cd29ceb0c45cea027db151fc77483 (127.30.185.129:36761)
I20260811 18:40:45.538559 4484 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:33667
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
I20260811 18:40:45.568076 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41299
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:42975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:45.909708 4875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:45.909986 4875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:45.910192 4875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:45.917564 4875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:45.917687 4875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:45.917892 4875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:40:45.928336 4875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:42975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41299
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:45.930035 4875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:45.931515 4875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:45.945559 4890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:45.945649 4887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:45.945600 4884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:45.950893 4875 server_base.cc:1034] running on GCE node
I20260811 18:40:45.951709 4875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:45.952757 4875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:45.954005 4875 hybrid_clock.cc:648] HybridClock initialized: now 1786473645953960 us; error 62 us; skew 500 ppm
I20260811 18:40:45.954363 4875 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:45 dist-test-slave-z8x9 krb5kdc[4403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473645, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:45.957862 4875 init.cc:377] Logged in from keytab as kudu/127.30.185.130@KRBTEST.COM (short username kudu)
I20260811 18:40:45.959442 4875 webserver.cc:460] Webserver started at http://127.30.185.130:35495/ using document root <none> and password file <none>
I20260811 18:40:45.960237 4875 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:45.960397 4875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:45.960729 4875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:45.963788 4875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "118f134492e7414eac2f145b49ff5729"
format_stamp: "Formatted at 2026-08-11 18:40:45 on dist-test-slave-z8x9"
server_key: "1b6268ce2c36563a7179cf44853b94fe"
server_key_iv: "30d9fdb0aaa93eb21daeab32916d77cd"
server_key_version: "encryptionkey@0"
I20260811 18:40:45.964640 4875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "118f134492e7414eac2f145b49ff5729"
format_stamp: "Formatted at 2026-08-11 18:40:45 on dist-test-slave-z8x9"
server_key: "1b6268ce2c36563a7179cf44853b94fe"
server_key_iv: "30d9fdb0aaa93eb21daeab32916d77cd"
server_key_version: "encryptionkey@0"
I20260811 18:40:45.970068 4875 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 18:40:45.973624 4902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:45.975023 4875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:40:45.975219 4875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "118f134492e7414eac2f145b49ff5729"
format_stamp: "Formatted at 2026-08-11 18:40:45 on dist-test-slave-z8x9"
server_key: "1b6268ce2c36563a7179cf44853b94fe"
server_key_iv: "30d9fdb0aaa93eb21daeab32916d77cd"
server_key_version: "encryptionkey@0"
I20260811 18:40:45.975445 4875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:46.003625 4875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:46.008534 4875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:46.008868 4875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:46.009967 4875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:40:46.011444 4875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:46.011555 4875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:46.011708 4875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:46.011799 4875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:46.049736 4875 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:42159
I20260811 18:40:46.049778 5111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:42159 every 8 connection(s)
I20260811 18:40:46.051067 4875 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:40:46.052945 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 4875
I20260811 18:40:46.053148 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:40:46.053730 31462 external_mini_cluster.cc:1428] Setting key 314842e4061c7c105b53e56eaf11bed4
Aug 11 18:40:46 dist-test-slave-z8x9 krb5kdc[4403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473645, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:46.075119 5116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:46.053655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:52999 (local address 127.30.185.190:41299)
0811 18:40:46.054873 (+ 1218us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:46.054881 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:46.055583 (+ 702us) server_negotiation.cc:407] Connection header received
0811 18:40:46.058390 (+ 2807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:46.058394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:46.058461 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:46.058556 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:46.060712 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:46.061505 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:46.064015 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:46.064432 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:46.064527 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:46.068715 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:46.068720 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:46.068723 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:46.068757 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:46.071573 (+ 2816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:46.072578 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:46.072582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:46.072584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:46.072711 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:46.073288 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:46.073292 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:46.073294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:46.073585 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:46.073699 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:46.074681 (+ 982us) server_negotiation.cc:299] Negotiation successful
0811 18:40:46.074856 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1053,"thread_start_us":955,"threads_started":1}
I20260811 18:40:46.076850 5117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:46.054185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41299 (local address 127.30.185.130:52999)
0811 18:40:46.055402 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:46.055457 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:46.058046 (+ 2589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:46.058847 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:46.058862 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:46.059667 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:46.060521 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:46.060550 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:46.061667 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:46.061675 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:46.063865 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:46.063883 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:46.064659 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:46.064669 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:46.065003 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:46.065842 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:46.065868 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:46.068460 (+ 2592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:46.071745 (+ 3285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:46.071761 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:46.071783 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:46.072458 (+ 675us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:46.072832 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:46.072840 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:46.072846 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:46.073144 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:46.073730 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:46.073742 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:46.074224 (+ 482us) client_negotiation.cc:769] Sending connection context
0811 18:40:46.074586 (+ 362us) client_negotiation.cc:240] Negotiation successful
0811 18:40:46.075209 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":940,"thread_start_us":742,"threads_started":1}
I20260811 18:40:46.078496 5112 heartbeater.cc:347] Connected to a master server at 127.30.185.190:41299
I20260811 18:40:46.078840 5112 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.30.185.131@KRBTEST.COM; defaulting to no policy
I20260811 18:40:46.079752 5112 heartbeater.cc:511] Master 127.30.185.190:41299 requested a full tablet report, sending...
I20260811 18:40:46.081384 4484 ts_manager.cc:194] Registered new tserver with Master: 118f134492e7414eac2f145b49ff5729 (127.30.185.130:42159)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.131@KRBTEST.COM" created.
I20260811 18:40:46.082280 4484 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:52999
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
I20260811 18:40:46.112068 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41299
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:42975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:46.454643 5123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:46.454929 5123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:46.455094 5123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:46.462464 5123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:46.462584 5123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:46.462757 5123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:40:46.472568 5123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:42975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41299
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:46.473958 5123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:46.475389 5123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:46.489924 5130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:46.489874 5132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:46.489874 5135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:46.490783 5123 server_base.cc:1034] running on GCE node
I20260811 18:40:46.491473 5123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:46.492448 5123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:46.493654 5123 hybrid_clock.cc:648] HybridClock initialized: now 1786473646493617 us; error 35 us; skew 500 ppm
I20260811 18:40:46.494001 5123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:46 dist-test-slave-z8x9 krb5kdc[4403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473646, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:46.497430 5123 init.cc:377] Logged in from keytab as kudu/127.30.185.131@KRBTEST.COM (short username kudu)
I20260811 18:40:46.498997 5123 webserver.cc:460] Webserver started at http://127.30.185.131:37577/ using document root <none> and password file <none>
I20260811 18:40:46.499701 5123 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:46.499864 5123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:46.500190 5123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:46.503283 5123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "127639bb8a7944d0afd004f1974a0cde"
format_stamp: "Formatted at 2026-08-11 18:40:46 on dist-test-slave-z8x9"
server_key: "dfa6007d710b7ed4c0a75aac73c22a48"
server_key_iv: "62704618be3da02e94f35a24e2676948"
server_key_version: "encryptionkey@0"
I20260811 18:40:46.504118 5123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "127639bb8a7944d0afd004f1974a0cde"
format_stamp: "Formatted at 2026-08-11 18:40:46 on dist-test-slave-z8x9"
server_key: "dfa6007d710b7ed4c0a75aac73c22a48"
server_key_iv: "62704618be3da02e94f35a24e2676948"
server_key_version: "encryptionkey@0"
I20260811 18:40:46.509841 5123 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 18:40:46.513394 5150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:46.514734 5123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:40:46.514926 5123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "127639bb8a7944d0afd004f1974a0cde"
format_stamp: "Formatted at 2026-08-11 18:40:46 on dist-test-slave-z8x9"
server_key: "dfa6007d710b7ed4c0a75aac73c22a48"
server_key_iv: "62704618be3da02e94f35a24e2676948"
server_key_version: "encryptionkey@0"
I20260811 18:40:46.515164 5123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:46.541787 4865 heartbeater.cc:503] Master 127.30.185.190:41299 was elected leader, sending a full tablet report...
I20260811 18:40:46.543870 5123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:46.548445 5123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:46.548766 5123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:46.549820 5123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:40:46.551194 5123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:46.551304 5123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:46.551461 5123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:46.551537 5123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:46.586642 5123 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:35639
I20260811 18:40:46.586664 5365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:35639 every 8 connection(s)
I20260811 18:40:46.588011 5123 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:40:46.597051 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 5123
I20260811 18:40:46.597322 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:40:46.597906 31462 external_mini_cluster.cc:1428] Setting key f58c2a575b2154feea8d708659e80062
Aug 11 18:40:46 dist-test-slave-z8x9 krb5kdc[4403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473646, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:46.608832 5370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:46.590793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:36067 (local address 127.30.185.190:41299)
0811 18:40:46.591224 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:46.591229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:46.592202 (+ 973us) server_negotiation.cc:407] Connection header received
0811 18:40:46.594064 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:46.594069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:46.594140 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:46.594251 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:46.595749 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:46.596627 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:46.598895 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:46.599435 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:46.599576 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:46.603197 (+ 3621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:46.603202 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:46.603205 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:46.603247 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:46.605985 (+ 2738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:46.606686 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:46.606690 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:46.606692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:46.606813 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:46.607214 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:46.607218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:46.607220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:46.607475 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:46.607592 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:46.608373 (+ 781us) server_negotiation.cc:299] Negotiation successful
0811 18:40:46.608539 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":211,"threads_started":1}
I20260811 18:40:46.609913 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:46.591363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41299 (local address 127.30.185.131:36067)
0811 18:40:46.592034 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:46.592073 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:46.593808 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:46.594460 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:46.594468 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:46.594973 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:46.595534 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:46.595552 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:46.596791 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:46.596798 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:46.598720 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:46.598738 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:46.599682 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:46.599690 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:46.599919 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:46.600629 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:46.600652 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:46.603001 (+ 2349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:46.606130 (+ 3129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:46.606137 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:46.606150 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:46.606576 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:46.606922 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:46.606925 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:46.606927 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:46.607111 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:46.607583 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:46.607589 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:46.607944 (+ 355us) client_negotiation.cc:769] Sending connection context
0811 18:40:46.608276 (+ 332us) client_negotiation.cc:240] Negotiation successful
0811 18:40:46.608711 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":275,"threads_started":1}
I20260811 18:40:46.611375 5368 heartbeater.cc:347] Connected to a master server at 127.30.185.190:41299
I20260811 18:40:46.611722 5368 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:46.612656 5368 heartbeater.cc:511] Master 127.30.185.190:41299 requested a full tablet report, sending...
I20260811 18:40:46.614252 4484 ts_manager.cc:194] Registered new tserver with Master: 127639bb8a7944d0afd004f1974a0cde (127.30.185.131:35639)
I20260811 18:40:46.615070 4484 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:36067
I20260811 18:40:46.625686 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 18:40:46 dist-test-slave-z8x9 krb5kdc[4403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473646, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 18:40:46 dist-test-slave-z8x9 krb5kdc[4403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473646, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:46.688761 5370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:46.674591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46728 (local address 127.30.185.190:41299)
0811 18:40:46.674849 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:46.674856 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:46.675218 (+ 362us) server_negotiation.cc:407] Connection header received
0811 18:40:46.675384 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:46.675388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:46.675456 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:46.675546 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:46.676865 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:46.677716 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:46.678981 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:46.679442 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:46.679541 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:46.683685 (+ 4144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:46.683690 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:46.683694 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:46.683728 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:46.686431 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:46.687023 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:46.687027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:46.687029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:46.687159 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:46.687494 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:46.687498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:46.687499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:46.687816 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:46.687995 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:46.688245 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 18:40:46.688425 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260811 18:40:46.689450 4484 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46728
I20260811 18:40:46.691082 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 4607
Aug 11 18:40:46 dist-test-slave-z8x9 krb5kdc[4403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473646, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.185.129@KRBTEST.COM
I20260811 18:40:46.707675 5385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:46.692124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48952 (local address 127.30.185.129:36761)
0811 18:40:46.692765 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:46.692772 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:46.692796 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:40:46.692851 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:46.693061 (+ 210us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:46.693384 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:46.693592 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:46.694843 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:46.695846 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:46.697042 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:46.697723 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:46.697863 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:46.701848 (+ 3985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:46.701860 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:46.701874 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:46.701911 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:46.704820 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:46.705566 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:46.705573 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:46.705575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:46.705701 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:46.706081 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:46.706086 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:46.706091 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:46.706436 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:46.706761 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:46.707055 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 18:40:46.707332 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":275,"threads_started":1}
W20260811 18:40:46.708940 4855 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48952
W20260811 18:40:46.710253 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 4607: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:40:46.710462 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 4607
I20260811 18:40:46.735569 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 4875
Aug 11 18:40:46 dist-test-slave-z8x9 krb5kdc[4403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473646, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.185.130@KRBTEST.COM
I20260811 18:40:46.752132 5386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:46.736519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33622 (local address 127.30.185.130:42159)
0811 18:40:46.737071 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:46.737078 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:46.737105 (+ 27us) server_negotiation.cc:407] Connection header received
0811 18:40:46.737243 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:46.737251 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:46.737481 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:46.737616 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:46.738823 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:46.739818 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:46.741047 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:46.741685 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:46.741801 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:46.746496 (+ 4695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:46.746509 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:46.746524 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:46.746558 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:46.749573 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:46.750213 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:46.750221 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:46.750223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:46.750379 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:46.750707 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:46.750711 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:46.750713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:46.751069 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:46.751331 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:46.751626 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 18:40:46.751842 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":287,"threads_started":1}
W20260811 18:40:46.753504 5102 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33622
W20260811 18:40:46.754562 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 4875: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:40:46.754745 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 4875
I20260811 18:40:46.784053 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 5123
Aug 11 18:40:46 dist-test-slave-z8x9 krb5kdc[4403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473646, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.185.131@KRBTEST.COM
I20260811 18:40:46.801362 5388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:46.785409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48748 (local address 127.30.185.131:35639)
0811 18:40:46.786082 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:46.786089 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:46.786117 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:40:46.786174 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:46.786181 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:46.786410 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:46.786536 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:46.787859 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:46.788911 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:46.790212 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:46.790793 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:46.790901 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:46.795326 (+ 4425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:46.795338 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:46.795366 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:46.795400 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:46.798455 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:46.799099 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:46.799105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:46.799107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:46.799224 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:46.799645 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:46.799649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:46.799651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:46.799989 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:46.800330 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:46.800650 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 18:40:46.800932 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":339,"threads_started":1}
W20260811 18:40:46.802687 5356 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48748
W20260811 18:40:46.803822 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 5123: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:40:46.803983 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 5123
I20260811 18:40:46.839318 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 4425
I20260811 18:40:46.840571 4546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:46.947868 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 4425
2026-08-11T18:40:46Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4190 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:40:47 dist-test-slave-z8x9 krb5kdc[5395](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:40:47 dist-test-slave-z8x9 krb5kdc[5395](info): set up 2 sockets
Aug 11 18:40:47 dist-test-slave-z8x9 krb5kdc[5395](info): commencing operation
krb5kdc: starting...
W20260811 18:40:48.334641 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.338s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:40:48 dist-test-slave-z8x9 krb5kdc[5395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:40:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:40:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:40:48.585752 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:41233
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:45033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:41233
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:48.922611 5414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:48.922967 5414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:48.923147 5414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:48.930415 5414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:40:48.930585 5414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:48.930702 5414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:48.930814 5414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:40:48.930907 5414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:40:48.941538 5414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:45033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:41233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:41233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:48.942958 5414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:48.944363 5414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:48.957087 5426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:48.957106 5429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:48.957078 5432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:48.958173 5414 server_base.cc:1034] running on GCE node
I20260811 18:40:48.959048 5414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:48.960569 5414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:48.961813 5414 hybrid_clock.cc:648] HybridClock initialized: now 1786473648961766 us; error 62 us; skew 500 ppm
I20260811 18:40:48.962267 5414 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:48 dist-test-slave-z8x9 krb5kdc[5395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473648, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:48.969693 5414 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:40:48.971418 5414 webserver.cc:460] Webserver started at http://127.30.185.190:37641/ using document root <none> and password file <none>
I20260811 18:40:48.972184 5414 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:48.972362 5414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:48.972707 5414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:48.975932 5414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "9a41ae83ccad46d1a35b03ff0b4e71eb"
format_stamp: "Formatted at 2026-08-11 18:40:48 on dist-test-slave-z8x9"
server_key: "9eefed276d910f5c9748b04106e43995"
server_key_iv: "84dc4bdb04fc0763da82109942f17e65"
server_key_version: "encryptionkey@0"
I20260811 18:40:48.976858 5414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "9a41ae83ccad46d1a35b03ff0b4e71eb"
format_stamp: "Formatted at 2026-08-11 18:40:48 on dist-test-slave-z8x9"
server_key: "9eefed276d910f5c9748b04106e43995"
server_key_iv: "84dc4bdb04fc0763da82109942f17e65"
server_key_version: "encryptionkey@0"
I20260811 18:40:48.982591 5414 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 18:40:48.986290 5445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:48.987659 5414 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:40:48.987862 5414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "9a41ae83ccad46d1a35b03ff0b4e71eb"
format_stamp: "Formatted at 2026-08-11 18:40:48 on dist-test-slave-z8x9"
server_key: "9eefed276d910f5c9748b04106e43995"
server_key_iv: "84dc4bdb04fc0763da82109942f17e65"
server_key_version: "encryptionkey@0"
I20260811 18:40:48.988070 5414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:49.011874 5414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:49.016607 5414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:49.016932 5414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:49.036180 5414 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:41233
I20260811 18:40:49.036209 5550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:41233 every 8 connection(s)
I20260811 18:40:49.037726 5414 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:40:49.040444 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 5414
I20260811 18:40:49.040652 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:40:49.041376 31462 external_mini_cluster.cc:1428] Setting key b4c5c70d47bb2576bd629a6b2cce13bf
I20260811 18:40:49.041672 5555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:49.056524 5555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a41ae83ccad46d1a35b03ff0b4e71eb: Bootstrap starting.
Aug 11 18:40:49 dist-test-slave-z8x9 krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:49.060608 5555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a41ae83ccad46d1a35b03ff0b4e71eb: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:49.061970 5555 log.cc:824] T 00000000000000000000000000000000 P 9a41ae83ccad46d1a35b03ff0b4e71eb: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:49.065430 5555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a41ae83ccad46d1a35b03ff0b4e71eb: No bootstrap required, opened a new log
I20260811 18:40:49.066860 5561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:49.043531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48144 (local address 127.30.185.190:41233)
0811 18:40:49.044402 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:49.044423 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:49.044474 (+ 51us) server_negotiation.cc:407] Connection header received
0811 18:40:49.046860 (+ 2386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:49.046891 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:49.047436 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:49.047901 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:49.049369 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:49.051404 (+ 2035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:49.052667 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:49.053794 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:49.053989 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:49.058047 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:49.058061 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:49.058085 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:49.058146 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:49.062274 (+ 4128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:49.063114 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:49.063120 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:49.063125 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:49.063291 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:49.063700 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:49.063704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:49.063707 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:49.064372 (+ 665us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:49.064668 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:49.065030 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 18:40:49.065495 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":346,"threads_started":1}
I20260811 18:40:49.073804 5555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a41ae83ccad46d1a35b03ff0b4e71eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a41ae83ccad46d1a35b03ff0b4e71eb" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41233 } }
I20260811 18:40:49.074144 5555 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a41ae83ccad46d1a35b03ff0b4e71eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:49.074219 5555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a41ae83ccad46d1a35b03ff0b4e71eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a41ae83ccad46d1a35b03ff0b4e71eb, State: Initialized, Role: FOLLOWER
I20260811 18:40:49.074765 5555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a41ae83ccad46d1a35b03ff0b4e71eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a41ae83ccad46d1a35b03ff0b4e71eb" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41233 } }
I20260811 18:40:49.074988 5555 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a41ae83ccad46d1a35b03ff0b4e71eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:40:49.075086 5555 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a41ae83ccad46d1a35b03ff0b4e71eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:40:49.075291 5555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a41ae83ccad46d1a35b03ff0b4e71eb [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:49.077755 5555 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a41ae83ccad46d1a35b03ff0b4e71eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a41ae83ccad46d1a35b03ff0b4e71eb" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41233 } }
I20260811 18:40:49.078195 5555 leader_election.cc:302] T 00000000000000000000000000000000 P 9a41ae83ccad46d1a35b03ff0b4e71eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a41ae83ccad46d1a35b03ff0b4e71eb; no voters:
I20260811 18:40:49.078750 5555 leader_election.cc:288] T 00000000000000000000000000000000 P 9a41ae83ccad46d1a35b03ff0b4e71eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:40:49.078907 5565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a41ae83ccad46d1a35b03ff0b4e71eb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:40:49.079696 5565 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a41ae83ccad46d1a35b03ff0b4e71eb [term 1 LEADER]: Becoming Leader. State: Replica: 9a41ae83ccad46d1a35b03ff0b4e71eb, State: Running, Role: LEADER
I20260811 18:40:49.080289 5565 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a41ae83ccad46d1a35b03ff0b4e71eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a41ae83ccad46d1a35b03ff0b4e71eb" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41233 } }
I20260811 18:40:49.080981 5555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a41ae83ccad46d1a35b03ff0b4e71eb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:40:49.083518 5566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a41ae83ccad46d1a35b03ff0b4e71eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a41ae83ccad46d1a35b03ff0b4e71eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a41ae83ccad46d1a35b03ff0b4e71eb" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41233 } } }
I20260811 18:40:49.083971 5566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a41ae83ccad46d1a35b03ff0b4e71eb [sys.catalog]: This master's current role is: LEADER
I20260811 18:40:49.083967 5569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a41ae83ccad46d1a35b03ff0b4e71eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a41ae83ccad46d1a35b03ff0b4e71eb. Latest consensus state: current_term: 1 leader_uuid: "9a41ae83ccad46d1a35b03ff0b4e71eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a41ae83ccad46d1a35b03ff0b4e71eb" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41233 } } }
I20260811 18:40:49.084266 5569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a41ae83ccad46d1a35b03ff0b4e71eb [sys.catalog]: This master's current role is: LEADER
I20260811 18:40:49.087064 5578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:40:49.092278 5578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:40:49.100716 5578 catalog_manager.cc:1347] Generated new cluster ID: c91687547b6548919be937b4e4a439e1
I20260811 18:40:49.100853 5578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:40:49.115877 5578 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:40:49.117274 5578 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:40:49.129047 5578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a41ae83ccad46d1a35b03ff0b4e71eb: Generated new TSK 0
I20260811 18:40:49.129931 5578 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.185.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
I20260811 18:40:49.195134 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41233
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:45033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:49.530578 5601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:49.530916 5601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:49.531087 5601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:49.538290 5601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:49.538431 5601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:49.538614 5601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:40:49.548168 5601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:45033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:49.549518 5601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:49.550910 5601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:49.564216 5607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:49.564157 5608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:49.564383 5611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:49.565286 5601 server_base.cc:1034] running on GCE node
I20260811 18:40:49.565908 5601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:49.566864 5601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:49.568097 5601 hybrid_clock.cc:648] HybridClock initialized: now 1786473649568057 us; error 54 us; skew 500 ppm
I20260811 18:40:49.568449 5601 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:49 dist-test-slave-z8x9 krb5kdc[5395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473649, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:49.571667 5601 init.cc:377] Logged in from keytab as kudu/127.30.185.129@KRBTEST.COM (short username kudu)
I20260811 18:40:49.573123 5601 webserver.cc:460] Webserver started at http://127.30.185.129:46529/ using document root <none> and password file <none>
I20260811 18:40:49.573853 5601 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:49.574007 5601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:49.574335 5601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:49.577286 5601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "29acb13ef57748b1b7757289fa9504d4"
format_stamp: "Formatted at 2026-08-11 18:40:49 on dist-test-slave-z8x9"
server_key: "5af0f825eef009d8c8109208ff1aea14"
server_key_iv: "ee8f478b4ae307f59c9feb1a198423cd"
server_key_version: "encryptionkey@0"
I20260811 18:40:49.578094 5601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "29acb13ef57748b1b7757289fa9504d4"
format_stamp: "Formatted at 2026-08-11 18:40:49 on dist-test-slave-z8x9"
server_key: "5af0f825eef009d8c8109208ff1aea14"
server_key_iv: "ee8f478b4ae307f59c9feb1a198423cd"
server_key_version: "encryptionkey@0"
I20260811 18:40:49.583446 5601 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 18:40:49.587064 5631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:49.588331 5601 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:40:49.588513 5601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "29acb13ef57748b1b7757289fa9504d4"
format_stamp: "Formatted at 2026-08-11 18:40:49 on dist-test-slave-z8x9"
server_key: "5af0f825eef009d8c8109208ff1aea14"
server_key_iv: "ee8f478b4ae307f59c9feb1a198423cd"
server_key_version: "encryptionkey@0"
I20260811 18:40:49.588737 5601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:49.612049 5601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:49.616477 5601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:49.616827 5601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:49.618072 5601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:40:49.619601 5601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:49.619733 5601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:49.619903 5601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:49.619987 5601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:49.656383 5601 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:45307
I20260811 18:40:49.656445 5857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:45307 every 8 connection(s)
I20260811 18:40:49.657938 5601 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:40:49.660334 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 5601
I20260811 18:40:49.660593 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:40:49.661535 31462 external_mini_cluster.cc:1428] Setting key 70dad20fc4da23f2e23ab822d530c03e
Aug 11 18:40:49 dist-test-slave-z8x9 krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473649, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:49.679924 5862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:49.660724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:40227 (local address 127.30.185.190:41233)
0811 18:40:49.661251 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:49.661257 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:49.662526 (+ 1269us) server_negotiation.cc:407] Connection header received
0811 18:40:49.664922 (+ 2396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:49.664930 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:49.665027 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:49.665152 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:49.666854 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:49.667965 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:49.669707 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:49.670423 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:49.670570 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:49.674145 (+ 3575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:49.674150 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:49.674153 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:49.674186 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:49.676756 (+ 2570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:49.677594 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:49.677599 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:49.677601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:49.677731 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:49.678182 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:49.678186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:49.678188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:49.678454 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:49.678612 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:49.679320 (+ 708us) server_negotiation.cc:299] Negotiation successful
0811 18:40:49.679557 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":236,"threads_started":1}
I20260811 18:40:49.680960 5863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:49.661352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41233 (local address 127.30.185.129:40227)
0811 18:40:49.662317 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:49.662376 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:49.664632 (+ 2256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:49.665480 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:49.665489 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:49.666052 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:49.666650 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:49.666674 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:49.668201 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:49.668206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:49.669557 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:49.669570 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:49.670638 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:49.670643 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:49.670848 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:49.671594 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:49.671618 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:49.673909 (+ 2291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:49.676906 (+ 2997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:49.676915 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:49.676930 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:49.677459 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:49.677856 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:49.677861 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:49.677865 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:49.678052 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:49.678585 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:49.678593 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:49.678937 (+ 344us) client_negotiation.cc:769] Sending connection context
0811 18:40:49.679203 (+ 266us) client_negotiation.cc:240] Negotiation successful
0811 18:40:49.679731 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":381,"threads_started":1}
I20260811 18:40:49.682588 5858 heartbeater.cc:347] Connected to a master server at 127.30.185.190:41233
I20260811 18:40:49.682967 5858 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:49.683924 5858 heartbeater.cc:511] Master 127.30.185.190:41233 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.185.130@KRBTEST.COM; defaulting to no policy
I20260811 18:40:49.686299 5481 ts_manager.cc:194] Registered new tserver with Master: 29acb13ef57748b1b7757289fa9504d4 (127.30.185.129:45307)
I20260811 18:40:49.687806 5481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:40227
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
I20260811 18:40:49.718202 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41233
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:45033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:50.071182 5868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:50.071468 5868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:50.071638 5868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:50.079559 5868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:50.079770 5868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:50.079970 5868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:40:50.090492 5868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:45033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:50.092041 5868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:50.093672 5868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:50.108148 5877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:50.108068 5884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:50.108050 5882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:50.109006 5868 server_base.cc:1034] running on GCE node
I20260811 18:40:50.109835 5868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:50.110982 5868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:50.112285 5868 hybrid_clock.cc:648] HybridClock initialized: now 1786473650112201 us; error 47 us; skew 500 ppm
I20260811 18:40:50.112686 5868 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:50 dist-test-slave-z8x9 krb5kdc[5395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473650, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:50.116323 5868 init.cc:377] Logged in from keytab as kudu/127.30.185.130@KRBTEST.COM (short username kudu)
I20260811 18:40:50.118072 5868 webserver.cc:460] Webserver started at http://127.30.185.130:38797/ using document root <none> and password file <none>
I20260811 18:40:50.118840 5868 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:50.118999 5868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:50.119340 5868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:50.122458 5868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "77b5e4363c45488c8817a0e6c6010194"
format_stamp: "Formatted at 2026-08-11 18:40:50 on dist-test-slave-z8x9"
server_key: "82751758202a7a1fab8eb78012d3b768"
server_key_iv: "6f3cf2d994436f8465084074da10a6ab"
server_key_version: "encryptionkey@0"
I20260811 18:40:50.123293 5868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "77b5e4363c45488c8817a0e6c6010194"
format_stamp: "Formatted at 2026-08-11 18:40:50 on dist-test-slave-z8x9"
server_key: "82751758202a7a1fab8eb78012d3b768"
server_key_iv: "6f3cf2d994436f8465084074da10a6ab"
server_key_version: "encryptionkey@0"
I20260811 18:40:50.129420 5868 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260811 18:40:50.133379 5897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:50.134856 5868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:40:50.135066 5868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "77b5e4363c45488c8817a0e6c6010194"
format_stamp: "Formatted at 2026-08-11 18:40:50 on dist-test-slave-z8x9"
server_key: "82751758202a7a1fab8eb78012d3b768"
server_key_iv: "6f3cf2d994436f8465084074da10a6ab"
server_key_version: "encryptionkey@0"
I20260811 18:40:50.135289 5868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:50.152087 5868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:50.156780 5868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:50.157047 5868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:50.158174 5868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:40:50.159618 5868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:50.159698 5868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:50.159791 5868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:50.159839 5868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:50.195917 5868 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:44515
I20260811 18:40:50.196194 6126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:44515 every 8 connection(s)
I20260811 18:40:50.197436 5868 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:40:50.202512 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 5868
I20260811 18:40:50.202701 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:40:50.203264 31462 external_mini_cluster.cc:1428] Setting key a85f3d720a00503581a49daa38f99d42
Aug 11 18:40:50 dist-test-slave-z8x9 krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473650, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:50.222344 6129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:50.199996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:36947 (local address 127.30.185.190:41233)
0811 18:40:50.200440 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:50.200445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:50.201508 (+ 1063us) server_negotiation.cc:407] Connection header received
0811 18:40:50.204054 (+ 2546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:50.204061 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:50.204168 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:50.204293 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:50.206096 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:50.206836 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:50.209465 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:50.209901 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:50.210002 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:50.214877 (+ 4875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:50.214882 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:50.214885 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:50.214924 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:50.218671 (+ 3747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:50.219522 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:50.219531 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:50.219537 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:50.219709 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:50.220152 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:50.220160 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:50.220166 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:50.220582 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:50.220759 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:50.221533 (+ 774us) server_negotiation.cc:299] Negotiation successful
0811 18:40:50.221815 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":229,"threads_started":1}
I20260811 18:40:50.222987 6130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:50.200563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41233 (local address 127.30.185.130:36947)
0811 18:40:50.201335 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:50.201388 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:50.203718 (+ 2330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:50.204557 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:50.204566 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:50.205076 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:50.205904 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:50.205931 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:50.207010 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:50.207019 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:50.209300 (+ 2281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:50.209320 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:50.210142 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:50.210151 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:50.210514 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:50.211314 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:50.211358 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:50.214568 (+ 3210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:50.218864 (+ 4296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:50.218872 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:50.218887 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:50.219361 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:50.219821 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:50.219824 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:50.219827 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:50.220028 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:50.220704 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:50.220710 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:50.221061 (+ 351us) client_negotiation.cc:769] Sending connection context
0811 18:40:50.221379 (+ 318us) client_negotiation.cc:240] Negotiation successful
0811 18:40:50.221805 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":278,"threads_started":1}
I20260811 18:40:50.224682 6127 heartbeater.cc:347] Connected to a master server at 127.30.185.190:41233
I20260811 18:40:50.225025 6127 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:50.225862 6127 heartbeater.cc:511] Master 127.30.185.190:41233 requested a full tablet report, sending...
I20260811 18:40:50.227543 5481 ts_manager.cc:194] Registered new tserver with Master: 77b5e4363c45488c8817a0e6c6010194 (127.30.185.130:44515)
I20260811 18:40:50.228404 5481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:36947
WARNING: no policy specified for kudu/127.30.185.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
I20260811 18:40:50.262475 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41233
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:45033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:50.598282 6137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:50.598623 6137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:50.598803 6137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:50.606170 6137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:50.606338 6137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:50.606525 6137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:40:50.616623 6137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:45033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:50.618065 6137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:50.619417 6137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:50.632884 6145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:50.633064 6148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:50.632856 6146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:50.633872 6137 server_base.cc:1034] running on GCE node
I20260811 18:40:50.634516 6137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:50.635550 6137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:50.636781 6137 hybrid_clock.cc:648] HybridClock initialized: now 1786473650636740 us; error 54 us; skew 500 ppm
I20260811 18:40:50.637112 6137 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:50 dist-test-slave-z8x9 krb5kdc[5395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473650, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:50.640549 6137 init.cc:377] Logged in from keytab as kudu/127.30.185.131@KRBTEST.COM (short username kudu)
I20260811 18:40:50.642045 6137 webserver.cc:460] Webserver started at http://127.30.185.131:32833/ using document root <none> and password file <none>
I20260811 18:40:50.642697 6137 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:50.642799 6137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:50.643061 6137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:50.646057 6137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "0004b19886eb4b36b368defec38ea4a4"
format_stamp: "Formatted at 2026-08-11 18:40:50 on dist-test-slave-z8x9"
server_key: "7a7d2baa5ff9640fa199cd88940eb9dc"
server_key_iv: "8164c92415c16c9f5e20e138e96a3943"
server_key_version: "encryptionkey@0"
I20260811 18:40:50.646822 6137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "0004b19886eb4b36b368defec38ea4a4"
format_stamp: "Formatted at 2026-08-11 18:40:50 on dist-test-slave-z8x9"
server_key: "7a7d2baa5ff9640fa199cd88940eb9dc"
server_key_iv: "8164c92415c16c9f5e20e138e96a3943"
server_key_version: "encryptionkey@0"
I20260811 18:40:50.652212 6137 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:40:50.655766 6164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:50.657517 6137 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:40:50.657771 6137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "0004b19886eb4b36b368defec38ea4a4"
format_stamp: "Formatted at 2026-08-11 18:40:50 on dist-test-slave-z8x9"
server_key: "7a7d2baa5ff9640fa199cd88940eb9dc"
server_key_iv: "8164c92415c16c9f5e20e138e96a3943"
server_key_version: "encryptionkey@0"
I20260811 18:40:50.657990 6137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:50.673053 6137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:50.677150 6137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:50.677524 6137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:50.678501 6137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:40:50.679828 6137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:50.679934 6137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:50.680080 6137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:50.680154 6137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:50.691054 5858 heartbeater.cc:503] Master 127.30.185.190:41233 was elected leader, sending a full tablet report...
I20260811 18:40:50.716061 6137 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:45729
I20260811 18:40:50.716110 6360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:45729 every 8 connection(s)
I20260811 18:40:50.717483 6137 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:40:50.727080 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 6137
I20260811 18:40:50.727270 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:40:50.727721 31462 external_mini_cluster.cc:1428] Setting key 5057018075d34e258bb3e7a2be2493f6
Aug 11 18:40:50 dist-test-slave-z8x9 krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473650, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:50.737740 6129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:50.720133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:34031 (local address 127.30.185.190:41233)
0811 18:40:50.720317 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:50.720321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:50.721515 (+ 1194us) server_negotiation.cc:407] Connection header received
0811 18:40:50.723327 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:50.723333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:50.723444 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:50.723572 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:50.724993 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:50.725788 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:50.727334 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:50.727786 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:50.727937 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:50.732088 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:50.732094 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:50.732099 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:50.732141 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:50.734632 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:50.735363 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:50.735369 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:50.735373 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:50.735506 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:50.735942 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:50.735948 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:50.735952 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:50.736240 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:50.736423 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:50.737103 (+ 680us) server_negotiation.cc:299] Negotiation successful
0811 18:40:50.737387 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 18:40:50.738682 6364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:50.720592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41233 (local address 127.30.185.131:34031)
0811 18:40:50.721353 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:50.721393 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:50.723058 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:50.723770 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:50.723778 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:50.724273 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:50.724826 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:50.724844 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:50.725927 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:50.725931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:50.727133 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:50.727145 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:50.728057 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:50.728067 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:50.728389 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:50.729156 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:50.729216 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:50.731879 (+ 2663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:50.734775 (+ 2896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:50.734783 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:50.734798 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:50.735219 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:50.735623 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:50.735627 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:50.735631 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:50.735826 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:50.736392 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:50.736401 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:50.736734 (+ 333us) client_negotiation.cc:769] Sending connection context
0811 18:40:50.737005 (+ 271us) client_negotiation.cc:240] Negotiation successful
0811 18:40:50.737597 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":286,"threads_started":1}
I20260811 18:40:50.739987 6361 heartbeater.cc:347] Connected to a master server at 127.30.185.190:41233
I20260811 18:40:50.740310 6361 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:50.741068 6361 heartbeater.cc:511] Master 127.30.185.190:41233 requested a full tablet report, sending...
I20260811 18:40:50.742538 5481 ts_manager.cc:194] Registered new tserver with Master: 0004b19886eb4b36b368defec38ea4a4 (127.30.185.131:45729)
I20260811 18:40:50.743522 5481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:34031
I20260811 18:40:50.743624 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 18:40:50 dist-test-slave-z8x9 krb5kdc[5395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:40:50 dist-test-slave-z8x9 krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.129@KRBTEST.COM
I20260811 18:40:50.803619 6375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:50.787850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41628 (local address 127.30.185.129:45307)
0811 18:40:50.788500 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:50.788507 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:50.788535 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:40:50.788597 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:50.788604 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:50.788815 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:50.789067 (+ 252us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:50.790536 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:50.791551 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:50.792851 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:50.793604 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:50.793718 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:50.797830 (+ 4112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:50.797842 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:50.797860 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:50.797900 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:50.800968 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:50.801690 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:50.801698 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:50.801700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:50.801827 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:50.802166 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:50.802169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:50.802171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:50.802534 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:50.802765 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:50.803001 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 18:40:50.803198 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":347,"threads_started":1}
Aug 11 18:40:50 dist-test-slave-z8x9 krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:50.822588 6129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:50.808633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48158 (local address 127.30.185.190:41233)
0811 18:40:50.808887 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:50.808893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:50.809077 (+ 184us) server_negotiation.cc:407] Connection header received
0811 18:40:50.809259 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:50.809263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:50.809339 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:50.809425 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:50.810695 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:50.811425 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:50.812588 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:50.812997 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:50.813093 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:50.817144 (+ 4051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:50.817150 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:50.817153 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:50.817248 (+ 95us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:50.819682 (+ 2434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:50.820409 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:50.820416 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:50.820419 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:50.820564 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:50.820980 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:50.820985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:50.820988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:50.821390 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:50.821664 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:50.822000 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 18:40:50.822238 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
W20260811 18:40:50.823181 5481 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48158
I20260811 18:40:50.824266 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 5601
I20260811 18:40:50.837461 6375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:50.825233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41644 (local address 127.30.185.129:45307)
0811 18:40:50.825457 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:50.825463 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:50.825700 (+ 237us) server_negotiation.cc:407] Connection header received
0811 18:40:50.825907 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:50.825912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:50.826007 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:50.826118 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:50.827359 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:50.828087 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:50.829384 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:50.829815 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:50.829962 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:50.832259 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:50.832264 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:50.832267 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:50.832300 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:50.834740 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:50.835434 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:50.835441 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:50.835444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:50.835576 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:50.835977 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:50.835982 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:50.835986 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:50.836251 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:50.836514 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:50.836846 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 18:40:50.837081 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:40:50.838078 5837 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:50.921790 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 5601
I20260811 18:40:50.945581 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 5868
Aug 11 18:40:50 dist-test-slave-z8x9 krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.130@KRBTEST.COM
I20260811 18:40:50.962412 6388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:50.946586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59032 (local address 127.30.185.130:44515)
0811 18:40:50.947207 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:50.947214 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:50.947243 (+ 29us) server_negotiation.cc:407] Connection header received
0811 18:40:50.947307 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:50.947313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:50.947526 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:50.947664 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:50.948924 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:50.950055 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:50.951280 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:50.951825 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:50.951923 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:50.956541 (+ 4618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:50.956553 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:50.956567 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:50.956599 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:50.959420 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:50.960118 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:50.960126 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:50.960128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:50.960253 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:50.960663 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:50.960669 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:50.960672 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:50.961039 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:50.961500 (+ 461us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:50.961764 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 18:40:50.962054 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":316,"threads_started":1}
I20260811 18:40:50.963795 6120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:51.046927 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 5868
I20260811 18:40:51.074131 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 6137
Aug 11 18:40:51 dist-test-slave-z8x9 krb5kdc[5395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.131@KRBTEST.COM
I20260811 18:40:51.090968 6397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:51.075029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44556 (local address 127.30.185.131:45729)
0811 18:40:51.075693 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:51.075700 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:51.075731 (+ 31us) server_negotiation.cc:407] Connection header received
0811 18:40:51.075794 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:51.075801 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:51.076016 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:51.076149 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:51.077536 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:51.078606 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:51.079808 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:51.080376 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:51.080469 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:51.085365 (+ 4896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:51.085378 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:51.085393 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:51.085436 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:51.088281 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:51.088941 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:51.088947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:51.088950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:51.089083 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:51.089511 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:51.089514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:51.089516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:51.089861 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:51.090112 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:51.090351 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 18:40:51.090588 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":339,"threads_started":1}
I20260811 18:40:51.092407 6354 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:51.178081 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 6137
I20260811 18:40:51.204396 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 5414
I20260811 18:40:51.205720 5545 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:51.312278 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 5414
2026-08-11T18:40:51Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4367 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:40:51 dist-test-slave-z8x9 krb5kdc[6408](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:40:51 dist-test-slave-z8x9 krb5kdc[6408](info): set up 2 sockets
Aug 11 18:40:51 dist-test-slave-z8x9 krb5kdc[6408](info): commencing operation
krb5kdc: starting...
W20260811 18:40:52.700162 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.336s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:40:52 dist-test-slave-z8x9 krb5kdc[6408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:40:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:40:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:40:52.957067 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:34965
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:34965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:53.302322 6432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:53.302656 6432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:53.302753 6432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:53.310115 6432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:40:53.310240 6432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:53.310295 6432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:53.310360 6432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:40:53.310410 6432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:40:53.320415 6432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:34965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:34965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:53.321736 6432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:53.323017 6432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:53.334749 6440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:53.334906 6439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:53.334775 6442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:53.335520 6432 server_base.cc:1034] running on GCE node
I20260811 18:40:53.336496 6432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:53.338025 6432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:53.339285 6432 hybrid_clock.cc:648] HybridClock initialized: now 1786473653339210 us; error 56 us; skew 500 ppm
I20260811 18:40:53.339651 6432 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:53 dist-test-slave-z8x9 krb5kdc[6408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473653, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:53.343062 6432 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:40:53.344208 6432 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:53.344310 6432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:53.344583 6432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:53.347690 6432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "98d069b0f6704c73b8af698ca705d89b"
format_stamp: "Formatted at 2026-08-11 18:40:53 on dist-test-slave-z8x9"
server_key: "1de3901d201dd44de17da5c131541bbc"
server_key_iv: "150a764b8403a5e525918f8d54ba080f"
server_key_version: "encryptionkey@0"
I20260811 18:40:53.348538 6432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "98d069b0f6704c73b8af698ca705d89b"
format_stamp: "Formatted at 2026-08-11 18:40:53 on dist-test-slave-z8x9"
server_key: "1de3901d201dd44de17da5c131541bbc"
server_key_iv: "150a764b8403a5e525918f8d54ba080f"
server_key_version: "encryptionkey@0"
I20260811 18:40:53.354238 6432 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 18:40:53.359172 6456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:53.361223 6432 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.007s sys 0.000s
I20260811 18:40:53.361459 6432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "98d069b0f6704c73b8af698ca705d89b"
format_stamp: "Formatted at 2026-08-11 18:40:53 on dist-test-slave-z8x9"
server_key: "1de3901d201dd44de17da5c131541bbc"
server_key_iv: "150a764b8403a5e525918f8d54ba080f"
server_key_version: "encryptionkey@0"
I20260811 18:40:53.361760 6432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:53.381894 6432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:53.386539 6432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:53.386852 6432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:53.405383 6432 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:34965
I20260811 18:40:53.405391 6549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:34965 every 8 connection(s)
I20260811 18:40:53.406368 6432 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:40:53.410168 6552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:53.413251 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 6432
I20260811 18:40:53.413479 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:40:53.414144 31462 external_mini_cluster.cc:1428] Setting key 37c9ba370a37fe67cb578feb1b7e3196
I20260811 18:40:53.424991 6552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b: Bootstrap starting.
I20260811 18:40:53.429253 6552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:53.430450 6552 log.cc:824] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:40:53 dist-test-slave-z8x9 krb5kdc[6408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:53.433924 6552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b: No bootstrap required, opened a new log
I20260811 18:40:53.440163 6556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:53.416338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59444 (local address 127.30.185.190:34965)
0811 18:40:53.417402 (+ 1064us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:53.417420 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:53.417476 (+ 56us) server_negotiation.cc:407] Connection header received
0811 18:40:53.420126 (+ 2650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:53.420164 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:53.420736 (+ 572us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:53.421258 (+ 522us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:53.422701 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:53.424810 (+ 2109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:53.426195 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:53.427492 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:53.427730 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:53.432404 (+ 4674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:53.432412 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:53.432427 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:53.432466 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:53.435620 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:53.436551 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:53.436558 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:53.436563 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:53.436753 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:53.437277 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:53.437281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:53.437283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:53.437882 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:53.438121 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:53.438645 (+ 524us) server_negotiation.cc:299] Negotiation successful
0811 18:40:53.439030 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":740,"thread_start_us":469,"threads_started":1}
I20260811 18:40:53.444433 6552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d069b0f6704c73b8af698ca705d89b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 34965 } }
I20260811 18:40:53.444859 6552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:53.444983 6552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98d069b0f6704c73b8af698ca705d89b, State: Initialized, Role: FOLLOWER
I20260811 18:40:53.445716 6552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d069b0f6704c73b8af698ca705d89b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 34965 } }
I20260811 18:40:53.445979 6552 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:40:53.446151 6552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:40:53.446358 6552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:53.449235 6552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d069b0f6704c73b8af698ca705d89b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 34965 } }
I20260811 18:40:53.449831 6552 leader_election.cc:302] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98d069b0f6704c73b8af698ca705d89b; no voters:
I20260811 18:40:53.450489 6552 leader_election.cc:288] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:40:53.450654 6563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:40:53.451496 6563 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b [term 1 LEADER]: Becoming Leader. State: Replica: 98d069b0f6704c73b8af698ca705d89b, State: Running, Role: LEADER
I20260811 18:40:53.452085 6563 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d069b0f6704c73b8af698ca705d89b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 34965 } }
I20260811 18:40:53.453127 6552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:40:53.455452 6573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98d069b0f6704c73b8af698ca705d89b. Latest consensus state: current_term: 1 leader_uuid: "98d069b0f6704c73b8af698ca705d89b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d069b0f6704c73b8af698ca705d89b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 34965 } } }
I20260811 18:40:53.455953 6573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b [sys.catalog]: This master's current role is: LEADER
I20260811 18:40:53.457635 6568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98d069b0f6704c73b8af698ca705d89b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d069b0f6704c73b8af698ca705d89b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 34965 } } }
I20260811 18:40:53.457937 6568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b [sys.catalog]: This master's current role is: LEADER
I20260811 18:40:53.458501 6582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:40:53.463718 6582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:40:53.472198 6582 catalog_manager.cc:1347] Generated new cluster ID: eb07c5c5ab5a49afa86edcd2c09b1db6
I20260811 18:40:53.472332 6582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:40:53.487947 6582 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:40:53.489495 6582 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:40:53.500115 6582 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b: Generated new TSK 0
I20260811 18:40:53.500969 6582 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.185.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
I20260811 18:40:53.569638 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:34965
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:53.908659 6614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:53.909008 6614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:53.909233 6614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:53.916648 6614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:53.916831 6614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:53.917032 6614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:40:53.927757 6614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:34965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:53.929292 6614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:53.930685 6614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:53.944408 6622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:53.944380 6623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:53.944334 6628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:53.945360 6614 server_base.cc:1034] running on GCE node
I20260811 18:40:53.946014 6614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:53.947005 6614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:53.948237 6614 hybrid_clock.cc:648] HybridClock initialized: now 1786473653948153 us; error 65 us; skew 500 ppm
I20260811 18:40:53.948567 6614 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:53 dist-test-slave-z8x9 krb5kdc[6408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473653, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:53.951897 6614 init.cc:377] Logged in from keytab as kudu/127.30.185.129@KRBTEST.COM (short username kudu)
I20260811 18:40:53.953336 6614 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:53.953500 6614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:53.953923 6614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:53.956982 6614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "839cadbf5ed9449c9440d310dcb7bd5a"
format_stamp: "Formatted at 2026-08-11 18:40:53 on dist-test-slave-z8x9"
server_key: "050aecbce17e03e815c9f7ca75d134ab"
server_key_iv: "f606ec485b4f3a12017957b7b3d947a6"
server_key_version: "encryptionkey@0"
I20260811 18:40:53.958046 6614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "839cadbf5ed9449c9440d310dcb7bd5a"
format_stamp: "Formatted at 2026-08-11 18:40:53 on dist-test-slave-z8x9"
server_key: "050aecbce17e03e815c9f7ca75d134ab"
server_key_iv: "f606ec485b4f3a12017957b7b3d947a6"
server_key_version: "encryptionkey@0"
I20260811 18:40:53.963610 6614 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:40:53.967381 6642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:53.968900 6614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:40:53.969053 6614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "839cadbf5ed9449c9440d310dcb7bd5a"
format_stamp: "Formatted at 2026-08-11 18:40:53 on dist-test-slave-z8x9"
server_key: "050aecbce17e03e815c9f7ca75d134ab"
server_key_iv: "f606ec485b4f3a12017957b7b3d947a6"
server_key_version: "encryptionkey@0"
I20260811 18:40:53.969368 6614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:54.000484 6614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:54.005146 6614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:54.005461 6614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:54.006434 6614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:40:54.007794 6614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:54.007884 6614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:54.007969 6614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:54.008029 6614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:54.043152 6614 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:43809
I20260811 18:40:54.043238 6878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:43809 every 8 connection(s)
I20260811 18:40:54.044082 6614 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:40:54.047314 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 6614
I20260811 18:40:54.047542 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:40:54.048439 31462 external_mini_cluster.cc:1428] Setting key 2f20c696cb5429c23fe3dde05ffb1e81
Aug 11 18:40:54 dist-test-slave-z8x9 krb5kdc[6408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473653, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:54.067130 6884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:54.046656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:36735 (local address 127.30.185.190:34965)
0811 18:40:54.047150 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:54.047155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:54.048214 (+ 1059us) server_negotiation.cc:407] Connection header received
0811 18:40:54.050592 (+ 2378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:54.050597 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:54.050695 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:54.050817 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:54.052320 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:54.053569 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:54.055276 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:54.055978 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:54.056129 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:54.059945 (+ 3816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:54.059954 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:54.059959 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:54.060002 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:54.063705 (+ 3703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:54.064734 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:54.064739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:54.064741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:54.064871 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:54.065371 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:54.065375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:54.065377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:54.065643 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:54.065758 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:54.066691 (+ 933us) server_negotiation.cc:299] Negotiation successful
0811 18:40:54.066859 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":278,"threads_started":1}
I20260811 18:40:54.068746 6885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:54.047207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:34965 (local address 127.30.185.129:36735)
0811 18:40:54.048046 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:54.048086 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:54.050223 (+ 2137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:54.051022 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:54.051031 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:54.051578 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:54.052129 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:54.052146 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:54.053719 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:54.053723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:54.055117 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:54.055130 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:54.056194 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:54.056199 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:54.056411 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:54.057156 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:54.057241 (+ 85us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:54.059716 (+ 2475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:54.063907 (+ 4191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:54.063920 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:54.063939 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:54.064604 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:54.064981 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:54.064989 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:54.064995 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:54.065263 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:54.065781 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:54.065792 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:54.066266 (+ 474us) client_negotiation.cc:769] Sending connection context
0811 18:40:54.066615 (+ 349us) client_negotiation.cc:240] Negotiation successful
0811 18:40:54.067210 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":411,"threads_started":1}
I20260811 18:40:54.071731 6879 heartbeater.cc:347] Connected to a master server at 127.30.185.190:34965
I20260811 18:40:54.072126 6879 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:54.073050 6879 heartbeater.cc:511] Master 127.30.185.190:34965 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.185.130@KRBTEST.COM; defaulting to no policy
I20260811 18:40:54.078576 6494 ts_manager.cc:194] Registered new tserver with Master: 839cadbf5ed9449c9440d310dcb7bd5a (127.30.185.129:43809)
I20260811 18:40:54.080920 6494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:36735
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
I20260811 18:40:54.121441 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:34965
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:54.460490 6891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:54.460785 6891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:54.461015 6891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:54.468596 6891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:54.468706 6891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:54.468925 6891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:40:54.478883 6891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:34965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:54.480299 6891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:54.481664 6891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:54.495050 6900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:54.494949 6901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:54.494984 6910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:54.495906 6891 server_base.cc:1034] running on GCE node
I20260811 18:40:54.496619 6891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:54.497699 6891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:54.498929 6891 hybrid_clock.cc:648] HybridClock initialized: now 1786473654498878 us; error 49 us; skew 500 ppm
I20260811 18:40:54.499251 6891 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:54 dist-test-slave-z8x9 krb5kdc[6408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473654, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:54.502648 6891 init.cc:377] Logged in from keytab as kudu/127.30.185.130@KRBTEST.COM (short username kudu)
I20260811 18:40:54.503894 6891 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:54.504033 6891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:54.504370 6891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:54.507588 6891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "5941907cd0ed41f3955d8eeccfab19f7"
format_stamp: "Formatted at 2026-08-11 18:40:54 on dist-test-slave-z8x9"
server_key: "c089da1f956979371deb1e3b71d76f2e"
server_key_iv: "e036bd8dd133126cdccb9a5d7f0f36a5"
server_key_version: "encryptionkey@0"
I20260811 18:40:54.508452 6891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "5941907cd0ed41f3955d8eeccfab19f7"
format_stamp: "Formatted at 2026-08-11 18:40:54 on dist-test-slave-z8x9"
server_key: "c089da1f956979371deb1e3b71d76f2e"
server_key_iv: "e036bd8dd133126cdccb9a5d7f0f36a5"
server_key_version: "encryptionkey@0"
I20260811 18:40:54.513849 6891 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 18:40:54.517364 6918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:54.519085 6891 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:40:54.519270 6891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "5941907cd0ed41f3955d8eeccfab19f7"
format_stamp: "Formatted at 2026-08-11 18:40:54 on dist-test-slave-z8x9"
server_key: "c089da1f956979371deb1e3b71d76f2e"
server_key_iv: "e036bd8dd133126cdccb9a5d7f0f36a5"
server_key_version: "encryptionkey@0"
I20260811 18:40:54.519515 6891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:54.543587 6891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:54.547878 6891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:54.548172 6891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:54.549264 6891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:40:54.550652 6891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:54.550765 6891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:54.550900 6891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:54.550992 6891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:54.586118 6891 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:39197
I20260811 18:40:54.586211 7125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:39197 every 8 connection(s)
I20260811 18:40:54.587031 6891 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:40:54.587388 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 6891
I20260811 18:40:54.587572 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:40:54.588195 31462 external_mini_cluster.cc:1428] Setting key eaa3f035bf43531d37c134115bfd4504
Aug 11 18:40:54 dist-test-slave-z8x9 krb5kdc[6408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473654, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:54.608429 7129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:54.589626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:41681 (local address 127.30.185.190:34965)
0811 18:40:54.590166 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:54.590172 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:54.591173 (+ 1001us) server_negotiation.cc:407] Connection header received
0811 18:40:54.593063 (+ 1890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:54.593068 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:54.593201 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:54.593373 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:54.594950 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:54.595736 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:54.597487 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:54.597949 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:54.598094 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:54.602415 (+ 4321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:54.602420 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:54.602424 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:54.602459 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:54.605156 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:54.606036 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:54.606042 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:54.606045 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:54.606186 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:54.606620 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:54.606628 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:54.606632 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:54.606921 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:54.607082 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:54.607786 (+ 704us) server_negotiation.cc:299] Negotiation successful
0811 18:40:54.608027 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":272,"threads_started":1}
I20260811 18:40:54.609543 7133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:54.590232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:34965 (local address 127.30.185.130:41681)
0811 18:40:54.590999 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:54.591038 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:54.592759 (+ 1721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:54.593598 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:54.593607 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:54.594160 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:54.594754 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:54.594772 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:54.595923 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:54.595927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:54.597317 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:54.597331 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:54.598210 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:54.598218 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:54.598466 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:54.599222 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:54.599247 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:54.602133 (+ 2886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:54.605399 (+ 3266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:54.605409 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:54.605428 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:54.605904 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:54.606309 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:54.606314 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:54.606317 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:54.606504 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:54.607053 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:54.607061 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:54.607417 (+ 356us) client_negotiation.cc:769] Sending connection context
0811 18:40:54.607688 (+ 271us) client_negotiation.cc:240] Negotiation successful
0811 18:40:54.608265 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":340,"threads_started":1}
I20260811 18:40:54.611335 7126 heartbeater.cc:347] Connected to a master server at 127.30.185.190:34965
I20260811 18:40:54.611661 7126 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:54.612324 7126 heartbeater.cc:511] Master 127.30.185.190:34965 requested a full tablet report, sending...
I20260811 18:40:54.613970 6494 ts_manager.cc:194] Registered new tserver with Master: 5941907cd0ed41f3955d8eeccfab19f7 (127.30.185.130:39197)
WARNING: no policy specified for kudu/127.30.185.131@KRBTEST.COM; defaulting to no policy
I20260811 18:40:54.614954 6494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:41681
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
I20260811 18:40:54.647935 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:34965
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:40:54.995080 7138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:54.995395 7138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:54.995494 7138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:55.003500 7138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:55.003717 7138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:55.003945 7138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:40:55.014356 7138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:34965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:55.015761 7138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:55.017450 7138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:55.030941 7147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:55.030988 7148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:55.031011 7150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:55.032132 7138 server_base.cc:1034] running on GCE node
I20260811 18:40:55.032778 7138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:55.033814 7138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:55.035014 7138 hybrid_clock.cc:648] HybridClock initialized: now 1786473655034949 us; error 57 us; skew 500 ppm
I20260811 18:40:55.035295 7138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:40:55 dist-test-slave-z8x9 krb5kdc[6408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473655, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:40:55.038625 7138 init.cc:377] Logged in from keytab as kudu/127.30.185.131@KRBTEST.COM (short username kudu)
I20260811 18:40:55.039731 7138 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:55.039834 7138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:55.040091 7138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:55.043764 7138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "8c31f044eee64c5581d66dd420517c6b"
format_stamp: "Formatted at 2026-08-11 18:40:55 on dist-test-slave-z8x9"
server_key: "d914fbd21726f8a9048ed707a0b53cca"
server_key_iv: "df470ecfb36386c3c13d15625d0f4dae"
server_key_version: "encryptionkey@0"
I20260811 18:40:55.044610 7138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "8c31f044eee64c5581d66dd420517c6b"
format_stamp: "Formatted at 2026-08-11 18:40:55 on dist-test-slave-z8x9"
server_key: "d914fbd21726f8a9048ed707a0b53cca"
server_key_iv: "df470ecfb36386c3c13d15625d0f4dae"
server_key_version: "encryptionkey@0"
I20260811 18:40:55.049804 7138 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 18:40:55.053421 7164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:55.054848 7138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:40:55.055011 7138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "8c31f044eee64c5581d66dd420517c6b"
format_stamp: "Formatted at 2026-08-11 18:40:55 on dist-test-slave-z8x9"
server_key: "d914fbd21726f8a9048ed707a0b53cca"
server_key_iv: "df470ecfb36386c3c13d15625d0f4dae"
server_key_version: "encryptionkey@0"
I20260811 18:40:55.055233 7138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:55.079466 7138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:55.083714 7138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:55.084021 7138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:55.084640 6879 heartbeater.cc:503] Master 127.30.185.190:34965 was elected leader, sending a full tablet report...
I20260811 18:40:55.085438 7138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:40:55.087059 7138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:55.087154 7138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:55.087240 7138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:55.087301 7138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:55.121959 7138 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:35235
I20260811 18:40:55.122057 7381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:35235 every 8 connection(s)
I20260811 18:40:55.122923 7138 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:40:55.128674 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 7138
I20260811 18:40:55.128890 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:40:55.129503 31462 external_mini_cluster.cc:1428] Setting key f33ed1f83d0cd2832ea4fd2d8a9f16e0
Aug 11 18:40:55 dist-test-slave-z8x9 krb5kdc[6408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473655, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:40:55.144022 7386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:55.125751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:39785 (local address 127.30.185.190:34965)
0811 18:40:55.126278 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:55.126283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:55.127078 (+ 795us) server_negotiation.cc:407] Connection header received
0811 18:40:55.129121 (+ 2043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:55.129129 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:55.129253 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:55.129386 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:55.130971 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.131963 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.133853 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.134278 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.134405 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:55.137923 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:55.137928 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:55.137931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:55.137966 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:55.140768 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:55.141578 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:55.141582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:55.141585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:55.141712 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:55.142153 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:55.142165 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:55.142168 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:55.142454 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:55.142641 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:55.143341 (+ 700us) server_negotiation.cc:299] Negotiation successful
0811 18:40:55.143600 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":280,"threads_started":1}
I20260811 18:40:55.145110 7390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:55.126173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:34965 (local address 127.30.185.131:39785)
0811 18:40:55.126897 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:55.126944 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:55.128735 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:55.129622 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:55.129635 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:55.130205 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:55.130801 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:55.130819 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.132121 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.132125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:55.133705 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:55.133717 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.134496 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.134501 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:55.134693 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:55.135351 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:40:55.135375 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:55.137700 (+ 2325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:55.140910 (+ 3210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:55.140917 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:55.140930 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:55.141473 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:55.141811 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:40:55.141814 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:40:55.141817 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:40:55.142015 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:40:55.142604 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:55.142614 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:55.142967 (+ 353us) client_negotiation.cc:769] Sending connection context
0811 18:40:55.143240 (+ 273us) client_negotiation.cc:240] Negotiation successful
0811 18:40:55.143808 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":292,"threads_started":1}
I20260811 18:40:55.146577 7384 heartbeater.cc:347] Connected to a master server at 127.30.185.190:34965
I20260811 18:40:55.146895 7384 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:55.147506 7384 heartbeater.cc:511] Master 127.30.185.190:34965 requested a full tablet report, sending...
I20260811 18:40:55.148953 6494 ts_manager.cc:194] Registered new tserver with Master: 8c31f044eee64c5581d66dd420517c6b (127.30.185.131:35235)
I20260811 18:40:55.149787 6494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:39785
I20260811 18:40:55.157895 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:40:55.178143 7386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:55.163024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59446 (local address 127.30.185.190:34965)
0811 18:40:55.163285 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:55.163295 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:55.163826 (+ 531us) server_negotiation.cc:407] Connection header received
0811 18:40:55.163901 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:55.163905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:55.163980 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:55.164082 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:55.165589 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.166588 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.168652 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.169418 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.169577 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:55.172261 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:55.172273 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:55.172276 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:55.172321 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:55.175250 (+ 2929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:55.175984 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:55.175991 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:55.175994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:55.176133 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:55.176530 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:55.176535 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:55.176538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:55.176892 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:55.177206 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:55.177563 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 18:40:55.177807 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 18:40:55.182724 6494 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:40:55.189870 6494 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:40:55.229141 7416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:55.203665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43014 (local address 127.30.185.130:39197)
0811 18:40:55.220261 (+ 16596us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:55.220269 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:55.221318 (+ 1049us) server_negotiation.cc:407] Connection header received
0811 18:40:55.221654 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:55.221661 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:55.221819 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:55.221951 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:55.222674 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.223866 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.226908 (+ 3042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.227883 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.228092 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:55.228285 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:55.228633 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 18:40:55.228813 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16424,"thread_start_us":16589,"threads_started":1}
I20260811 18:40:55.229563 7412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:55.202416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41002 (local address 127.30.185.131:35235)
0811 18:40:55.220953 (+ 18537us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:55.220961 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:55.220996 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:40:55.221096 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:55.221102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:55.221375 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:55.222296 (+ 921us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:55.222634 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.224126 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.226958 (+ 2832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.228390 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.228588 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:55.228734 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:55.228977 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 18:40:55.229155 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18355,"thread_start_us":18189,"threads_started":1}
I20260811 18:40:55.230201 7415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:55.203164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33046 (local address 127.30.185.129:43809)
0811 18:40:55.219926 (+ 16762us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:55.219938 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:55.219976 (+ 38us) server_negotiation.cc:407] Connection header received
0811 18:40:55.220100 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:55.220109 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:55.220410 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:55.220674 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:55.221600 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.223456 (+ 1856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.226707 (+ 3251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.228261 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.228479 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:55.228664 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:55.229846 (+ 1182us) server_negotiation.cc:299] Negotiation successful
0811 18:40:55.229974 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16576,"thread_start_us":16432,"threads_started":1}
I20260811 18:40:55.230647 7411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:55.202280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:43809 (local address 127.0.0.1:33046)
0811 18:40:55.217323 (+ 15043us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:55.217357 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:55.217521 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:55.220703 (+ 3182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:55.220711 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:55.220754 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:55.221349 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:55.221364 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.223623 (+ 2259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.223628 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:55.226501 (+ 2873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:55.226517 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.228842 (+ 2325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.228847 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:55.229045 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:55.229079 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 18:40:55.229161 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 18:40:55.229348 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14809,"mutex_wait_us":112,"thread_start_us":14581,"threads_started":1}
I20260811 18:40:55.231145 7410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:55.202159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:35235 (local address 127.0.0.1:41002)
0811 18:40:55.213928 (+ 11769us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:55.214002 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:55.214275 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:55.221597 (+ 7322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:55.221602 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:55.221628 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:55.222092 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:55.222105 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.224310 (+ 2205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.224317 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:55.226780 (+ 2463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:55.226793 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.228594 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.228601 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:55.228779 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:55.228810 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 18:40:55.228909 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 18:40:55.229122 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11366,"mutex_wait_us":139,"thread_start_us":12404,"threads_started":1}
I20260811 18:40:55.232087 7414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:55.202698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:39197 (local address 127.0.0.1:43014)
0811 18:40:55.221131 (+ 18433us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:55.221158 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:55.221441 (+ 283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:55.222154 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:55.222157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:55.222176 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:55.222439 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:55.222446 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.224021 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.224027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:55.226726 (+ 2699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:55.226743 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.228093 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.228100 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:55.228320 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:55.228380 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 18:40:55.229933 (+ 1553us) client_negotiation.cc:240] Negotiation successful
0811 18:40:55.230052 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18306,"mutex_wait_us":138,"thread_start_us":16685,"threads_started":1}
I20260811 18:40:55.234539 6750 tablet_service.cc:1467] Processing CreateTablet for tablet 467e8dbe57474750b5f015d5ec0ada09 (DEFAULT_TABLE table=test-table [id=ad1258cd196e446dae2f91b297ae3775]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:55.234798 7267 tablet_service.cc:1467] Processing CreateTablet for tablet 467e8dbe57474750b5f015d5ec0ada09 (DEFAULT_TABLE table=test-table [id=ad1258cd196e446dae2f91b297ae3775]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:55.235898 6750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 467e8dbe57474750b5f015d5ec0ada09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:55.235648 7007 tablet_service.cc:1467] Processing CreateTablet for tablet 467e8dbe57474750b5f015d5ec0ada09 (DEFAULT_TABLE table=test-table [id=ad1258cd196e446dae2f91b297ae3775]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:55.236236 7267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 467e8dbe57474750b5f015d5ec0ada09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:55.236956 7007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 467e8dbe57474750b5f015d5ec0ada09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:55.247530 7417 tablet_bootstrap.cc:492] T 467e8dbe57474750b5f015d5ec0ada09 P 839cadbf5ed9449c9440d310dcb7bd5a: Bootstrap starting.
I20260811 18:40:55.251259 7417 tablet_bootstrap.cc:654] T 467e8dbe57474750b5f015d5ec0ada09 P 839cadbf5ed9449c9440d310dcb7bd5a: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:55.252292 7420 tablet_bootstrap.cc:492] T 467e8dbe57474750b5f015d5ec0ada09 P 8c31f044eee64c5581d66dd420517c6b: Bootstrap starting.
I20260811 18:40:55.252633 7417 log.cc:824] T 467e8dbe57474750b5f015d5ec0ada09 P 839cadbf5ed9449c9440d310dcb7bd5a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:55.254161 7419 tablet_bootstrap.cc:492] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7: Bootstrap starting.
I20260811 18:40:55.256779 7420 tablet_bootstrap.cc:654] T 467e8dbe57474750b5f015d5ec0ada09 P 8c31f044eee64c5581d66dd420517c6b: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:55.257481 7419 tablet_bootstrap.cc:654] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:55.258049 7417 tablet_bootstrap.cc:492] T 467e8dbe57474750b5f015d5ec0ada09 P 839cadbf5ed9449c9440d310dcb7bd5a: No bootstrap required, opened a new log
I20260811 18:40:55.258165 7420 log.cc:824] T 467e8dbe57474750b5f015d5ec0ada09 P 8c31f044eee64c5581d66dd420517c6b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:55.258348 7417 ts_tablet_manager.cc:1397] T 467e8dbe57474750b5f015d5ec0ada09 P 839cadbf5ed9449c9440d310dcb7bd5a: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.008s
I20260811 18:40:55.258636 7419 log.cc:824] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:55.269663 7419 tablet_bootstrap.cc:492] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7: No bootstrap required, opened a new log
I20260811 18:40:55.270041 7419 ts_tablet_manager.cc:1397] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.009s
I20260811 18:40:55.270143 7420 tablet_bootstrap.cc:492] T 467e8dbe57474750b5f015d5ec0ada09 P 8c31f044eee64c5581d66dd420517c6b: No bootstrap required, opened a new log
I20260811 18:40:55.270421 7420 ts_tablet_manager.cc:1397] T 467e8dbe57474750b5f015d5ec0ada09 P 8c31f044eee64c5581d66dd420517c6b: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.007s
I20260811 18:40:55.271049 7417 raft_consensus.cc:348] T 467e8dbe57474750b5f015d5ec0ada09 P 839cadbf5ed9449c9440d310dcb7bd5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c31f044eee64c5581d66dd420517c6b" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 35235 } } peers { permanent_uuid: "839cadbf5ed9449c9440d310dcb7bd5a" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 43809 } } peers { permanent_uuid: "5941907cd0ed41f3955d8eeccfab19f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39197 } }
I20260811 18:40:55.271519 7417 raft_consensus.cc:374] T 467e8dbe57474750b5f015d5ec0ada09 P 839cadbf5ed9449c9440d310dcb7bd5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:55.271632 7417 raft_consensus.cc:729] T 467e8dbe57474750b5f015d5ec0ada09 P 839cadbf5ed9449c9440d310dcb7bd5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 839cadbf5ed9449c9440d310dcb7bd5a, State: Initialized, Role: FOLLOWER
I20260811 18:40:55.272289 7417 consensus_queue.cc:260] T 467e8dbe57474750b5f015d5ec0ada09 P 839cadbf5ed9449c9440d310dcb7bd5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c31f044eee64c5581d66dd420517c6b" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 35235 } } peers { permanent_uuid: "839cadbf5ed9449c9440d310dcb7bd5a" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 43809 } } peers { permanent_uuid: "5941907cd0ed41f3955d8eeccfab19f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39197 } }
I20260811 18:40:55.278178 7419 raft_consensus.cc:348] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c31f044eee64c5581d66dd420517c6b" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 35235 } } peers { permanent_uuid: "839cadbf5ed9449c9440d310dcb7bd5a" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 43809 } } peers { permanent_uuid: "5941907cd0ed41f3955d8eeccfab19f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39197 } }
I20260811 18:40:55.278545 7419 raft_consensus.cc:374] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:55.278657 7419 raft_consensus.cc:729] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5941907cd0ed41f3955d8eeccfab19f7, State: Initialized, Role: FOLLOWER
I20260811 18:40:55.278703 7417 ts_tablet_manager.cc:1428] T 467e8dbe57474750b5f015d5ec0ada09 P 839cadbf5ed9449c9440d310dcb7bd5a: Time spent starting tablet: real 0.020s user 0.009s sys 0.011s
I20260811 18:40:55.279263 7419 consensus_queue.cc:260] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c31f044eee64c5581d66dd420517c6b" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 35235 } } peers { permanent_uuid: "839cadbf5ed9449c9440d310dcb7bd5a" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 43809 } } peers { permanent_uuid: "5941907cd0ed41f3955d8eeccfab19f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39197 } }
I20260811 18:40:55.280493 7126 heartbeater.cc:503] Master 127.30.185.190:34965 was elected leader, sending a full tablet report...
I20260811 18:40:55.281010 7419 ts_tablet_manager.cc:1428] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260811 18:40:55.283416 7420 raft_consensus.cc:348] T 467e8dbe57474750b5f015d5ec0ada09 P 8c31f044eee64c5581d66dd420517c6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c31f044eee64c5581d66dd420517c6b" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 35235 } } peers { permanent_uuid: "839cadbf5ed9449c9440d310dcb7bd5a" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 43809 } } peers { permanent_uuid: "5941907cd0ed41f3955d8eeccfab19f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39197 } }
I20260811 18:40:55.283787 7420 raft_consensus.cc:374] T 467e8dbe57474750b5f015d5ec0ada09 P 8c31f044eee64c5581d66dd420517c6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:55.283864 7420 raft_consensus.cc:729] T 467e8dbe57474750b5f015d5ec0ada09 P 8c31f044eee64c5581d66dd420517c6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c31f044eee64c5581d66dd420517c6b, State: Initialized, Role: FOLLOWER
I20260811 18:40:55.284447 7420 consensus_queue.cc:260] T 467e8dbe57474750b5f015d5ec0ada09 P 8c31f044eee64c5581d66dd420517c6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c31f044eee64c5581d66dd420517c6b" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 35235 } } peers { permanent_uuid: "839cadbf5ed9449c9440d310dcb7bd5a" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 43809 } } peers { permanent_uuid: "5941907cd0ed41f3955d8eeccfab19f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39197 } }
I20260811 18:40:55.285705 7384 heartbeater.cc:503] Master 127.30.185.190:34965 was elected leader, sending a full tablet report...
I20260811 18:40:55.286228 7420 ts_tablet_manager.cc:1428] T 467e8dbe57474750b5f015d5ec0ada09 P 8c31f044eee64c5581d66dd420517c6b: Time spent starting tablet: real 0.016s user 0.013s sys 0.001s
W20260811 18:40:55.296543 6883 tablet.cc:2369] T 467e8dbe57474750b5f015d5ec0ada09 P 839cadbf5ed9449c9440d310dcb7bd5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:40:55.339202 7127 tablet.cc:2369] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:40:55.374742 7385 tablet.cc:2369] T 467e8dbe57474750b5f015d5ec0ada09 P 8c31f044eee64c5581d66dd420517c6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:40:55.504179 7427 raft_consensus.cc:482] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:40:55.504580 7427 raft_consensus.cc:504] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c31f044eee64c5581d66dd420517c6b" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 35235 } } peers { permanent_uuid: "839cadbf5ed9449c9440d310dcb7bd5a" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 43809 } } peers { permanent_uuid: "5941907cd0ed41f3955d8eeccfab19f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39197 } }
I20260811 18:40:55.506436 7427 leader_election.cc:288] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c31f044eee64c5581d66dd420517c6b (127.30.185.131:35235), 839cadbf5ed9449c9440d310dcb7bd5a (127.30.185.129:43809)
I20260811 18:40:55.513195 7415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:55.507068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:40083 (local address 127.30.185.129:43809)
0811 18:40:55.507258 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:55.507262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:55.507938 (+ 676us) server_negotiation.cc:407] Connection header received
0811 18:40:55.508193 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:55.508199 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:55.508271 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:55.508378 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:55.508908 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.509644 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.511442 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.512232 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.512430 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:55.512507 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:55.512782 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 18:40:55.512922 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 18:40:55.513617 7430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:55.507056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:43809 (local address 127.30.185.130:40083)
0811 18:40:55.507800 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:55.507818 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:55.508024 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:55.508401 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:55.508406 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:55.508433 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:55.508762 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:55.508771 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.509786 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.509791 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:55.511271 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:55.511284 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.512465 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.512471 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:55.512614 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:55.512638 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:40:55.512746 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 18:40:55.512880 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"mutex_wait_us":3,"thread_start_us":415,"threads_started":1}
I20260811 18:40:55.513705 7412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:55.507056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:34265 (local address 127.30.185.131:35235)
0811 18:40:55.507288 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:55.507297 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:55.508439 (+ 1142us) server_negotiation.cc:407] Connection header received
0811 18:40:55.508656 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:55.508666 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:55.508724 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:55.508826 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:40:55.509438 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.510183 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.511914 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.512699 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.512890 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:55.512967 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:55.513305 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 18:40:55.513430 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 18:40:55.513948 7432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:55.507056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:35235 (local address 127.30.185.130:34265)
0811 18:40:55.508317 (+ 1261us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:55.508333 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:55.508499 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:55.508838 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:55.508843 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:55.508865 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:40:55.509281 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:55.509292 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.510344 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.510350 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:55.511745 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:55.511757 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.512929 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.512935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:55.513115 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:55.513140 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:40:55.513278 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 18:40:55.513401 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1150,"mutex_wait_us":254,"thread_start_us":607,"threads_started":1}
I20260811 18:40:55.514823 6790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467e8dbe57474750b5f015d5ec0ada09" candidate_uuid: "5941907cd0ed41f3955d8eeccfab19f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "839cadbf5ed9449c9440d310dcb7bd5a" is_pre_election: true
I20260811 18:40:55.514956 7303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467e8dbe57474750b5f015d5ec0ada09" candidate_uuid: "5941907cd0ed41f3955d8eeccfab19f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c31f044eee64c5581d66dd420517c6b" is_pre_election: true
I20260811 18:40:55.515462 6790 raft_consensus.cc:2454] T 467e8dbe57474750b5f015d5ec0ada09 P 839cadbf5ed9449c9440d310dcb7bd5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5941907cd0ed41f3955d8eeccfab19f7 in term 0.
I20260811 18:40:55.515558 7303 raft_consensus.cc:2454] T 467e8dbe57474750b5f015d5ec0ada09 P 8c31f044eee64c5581d66dd420517c6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5941907cd0ed41f3955d8eeccfab19f7 in term 0.
I20260811 18:40:55.516237 6925 leader_election.cc:302] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5941907cd0ed41f3955d8eeccfab19f7, 839cadbf5ed9449c9440d310dcb7bd5a; no voters:
I20260811 18:40:55.516681 7427 raft_consensus.cc:2781] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:40:55.516847 7427 raft_consensus.cc:482] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:40:55.516970 7427 raft_consensus.cc:3037] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:55.519218 7427 raft_consensus.cc:504] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c31f044eee64c5581d66dd420517c6b" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 35235 } } peers { permanent_uuid: "839cadbf5ed9449c9440d310dcb7bd5a" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 43809 } } peers { permanent_uuid: "5941907cd0ed41f3955d8eeccfab19f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39197 } }
I20260811 18:40:55.519841 7427 leader_election.cc:288] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7 [CANDIDATE]: Term 1 election: Requested vote from peers 8c31f044eee64c5581d66dd420517c6b (127.30.185.131:35235), 839cadbf5ed9449c9440d310dcb7bd5a (127.30.185.129:43809)
I20260811 18:40:55.520319 7303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467e8dbe57474750b5f015d5ec0ada09" candidate_uuid: "5941907cd0ed41f3955d8eeccfab19f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c31f044eee64c5581d66dd420517c6b"
I20260811 18:40:55.520325 6790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467e8dbe57474750b5f015d5ec0ada09" candidate_uuid: "5941907cd0ed41f3955d8eeccfab19f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "839cadbf5ed9449c9440d310dcb7bd5a"
I20260811 18:40:55.520529 7303 raft_consensus.cc:3037] T 467e8dbe57474750b5f015d5ec0ada09 P 8c31f044eee64c5581d66dd420517c6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:55.520543 6790 raft_consensus.cc:3037] T 467e8dbe57474750b5f015d5ec0ada09 P 839cadbf5ed9449c9440d310dcb7bd5a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:55.522840 6790 raft_consensus.cc:2454] T 467e8dbe57474750b5f015d5ec0ada09 P 839cadbf5ed9449c9440d310dcb7bd5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5941907cd0ed41f3955d8eeccfab19f7 in term 1.
I20260811 18:40:55.522842 7303 raft_consensus.cc:2454] T 467e8dbe57474750b5f015d5ec0ada09 P 8c31f044eee64c5581d66dd420517c6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5941907cd0ed41f3955d8eeccfab19f7 in term 1.
I20260811 18:40:55.523250 6925 leader_election.cc:302] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5941907cd0ed41f3955d8eeccfab19f7, 839cadbf5ed9449c9440d310dcb7bd5a; no voters:
I20260811 18:40:55.523581 7427 raft_consensus.cc:2781] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:40:55.524065 7427 raft_consensus.cc:686] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7 [term 1 LEADER]: Becoming Leader. State: Replica: 5941907cd0ed41f3955d8eeccfab19f7, State: Running, Role: LEADER
I20260811 18:40:55.524610 7427 consensus_queue.cc:237] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c31f044eee64c5581d66dd420517c6b" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 35235 } } peers { permanent_uuid: "839cadbf5ed9449c9440d310dcb7bd5a" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 43809 } } peers { permanent_uuid: "5941907cd0ed41f3955d8eeccfab19f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39197 } }
I20260811 18:40:55.529615 6493 catalog_manager.cc:5495] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5941907cd0ed41f3955d8eeccfab19f7 (127.30.185.130). New cstate: current_term: 1 leader_uuid: "5941907cd0ed41f3955d8eeccfab19f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c31f044eee64c5581d66dd420517c6b" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 35235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "839cadbf5ed9449c9440d310dcb7bd5a" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 43809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5941907cd0ed41f3955d8eeccfab19f7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39197 } health_report { overall_health: HEALTHY } } }
I20260811 18:40:55.600409 7416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:55.595074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43026 (local address 127.30.185.130:39197)
0811 18:40:55.595313 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:55.595322 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:55.595343 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:40:55.595440 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:55.595444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:55.595530 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:55.595694 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:40:55.596070 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.596839 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.598254 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.598688 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.598823 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:55.599307 (+ 484us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:40:55.599788 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:40:55.599942 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:55.600044 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 18:40:55.600127 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 18:40:55.609126 7303 raft_consensus.cc:1261] T 467e8dbe57474750b5f015d5ec0ada09 P 8c31f044eee64c5581d66dd420517c6b [term 1 FOLLOWER]: Refusing update from remote peer 5941907cd0ed41f3955d8eeccfab19f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:40:55.609126 6790 raft_consensus.cc:1261] T 467e8dbe57474750b5f015d5ec0ada09 P 839cadbf5ed9449c9440d310dcb7bd5a [term 1 FOLLOWER]: Refusing update from remote peer 5941907cd0ed41f3955d8eeccfab19f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:40:55.610263 7427 consensus_queue.cc:1035] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c31f044eee64c5581d66dd420517c6b" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 35235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:40:55.610595 7433 consensus_queue.cc:1035] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "839cadbf5ed9449c9440d310dcb7bd5a" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 43809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:40:55.622649 7441 mvcc.cc:205] Tried to move back new op lower bound from 7317396093363220480 to 7317396093033021440. Current Snapshot: MvccSnapshot[applied={T|T < 7317396093363220480}]
I20260811 18:40:55.638988 6493 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:40:55.639293 6493 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:40:55.643250 6493 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98d069b0f6704c73b8af698ca705d89b: Sending DeleteTablet for 3 replicas of tablet 467e8dbe57474750b5f015d5ec0ada09
I20260811 18:40:55.644550 7267 tablet_service.cc:1514] Processing DeleteTablet for tablet 467e8dbe57474750b5f015d5ec0ada09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:55 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:41002
I20260811 18:40:55.644551 6750 tablet_service.cc:1514] Processing DeleteTablet for tablet 467e8dbe57474750b5f015d5ec0ada09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:55 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:33046
I20260811 18:40:55.644563 7007 tablet_service.cc:1514] Processing DeleteTablet for tablet 467e8dbe57474750b5f015d5ec0ada09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:55 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:43014
I20260811 18:40:55.647271 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 6614
I20260811 18:40:55.655867 7463 tablet_replica.cc:330] T 467e8dbe57474750b5f015d5ec0ada09 P 8c31f044eee64c5581d66dd420517c6b: stopping tablet replica
I20260811 18:40:55.656605 7463 raft_consensus.cc:2229] T 467e8dbe57474750b5f015d5ec0ada09 P 8c31f044eee64c5581d66dd420517c6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:40:55.657526 7463 raft_consensus.cc:2258] T 467e8dbe57474750b5f015d5ec0ada09 P 8c31f044eee64c5581d66dd420517c6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:40:55.658459 7461 tablet_replica.cc:330] T 467e8dbe57474750b5f015d5ec0ada09 P 839cadbf5ed9449c9440d310dcb7bd5a: stopping tablet replica
I20260811 18:40:55.659001 7461 raft_consensus.cc:2229] T 467e8dbe57474750b5f015d5ec0ada09 P 839cadbf5ed9449c9440d310dcb7bd5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:40:55.659910 7461 raft_consensus.cc:2258] T 467e8dbe57474750b5f015d5ec0ada09 P 839cadbf5ed9449c9440d310dcb7bd5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:40:55.660085 7463 ts_tablet_manager.cc:1905] T 467e8dbe57474750b5f015d5ec0ada09 P 8c31f044eee64c5581d66dd420517c6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:40:55.662611 7461 ts_tablet_manager.cc:1905] T 467e8dbe57474750b5f015d5ec0ada09 P 839cadbf5ed9449c9440d310dcb7bd5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:40:55.668234 7463 ts_tablet_manager.cc:1918] T 467e8dbe57474750b5f015d5ec0ada09 P 8c31f044eee64c5581d66dd420517c6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:40:55.668437 7463 log.cc:1196] T 467e8dbe57474750b5f015d5ec0ada09 P 8c31f044eee64c5581d66dd420517c6b: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/467e8dbe57474750b5f015d5ec0ada09
I20260811 18:40:55.669056 7463 ts_tablet_manager.cc:1939] T 467e8dbe57474750b5f015d5ec0ada09 P 8c31f044eee64c5581d66dd420517c6b: Deleting consensus metadata
I20260811 18:40:55.670023 7461 ts_tablet_manager.cc:1918] T 467e8dbe57474750b5f015d5ec0ada09 P 839cadbf5ed9449c9440d310dcb7bd5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:40:55.670280 7461 log.cc:1196] T 467e8dbe57474750b5f015d5ec0ada09 P 839cadbf5ed9449c9440d310dcb7bd5a: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-0/wal/wals/467e8dbe57474750b5f015d5ec0ada09
I20260811 18:40:55.670938 7461 ts_tablet_manager.cc:1939] T 467e8dbe57474750b5f015d5ec0ada09 P 839cadbf5ed9449c9440d310dcb7bd5a: Deleting consensus metadata
I20260811 18:40:55.671587 6458 catalog_manager.cc:4841] TS 8c31f044eee64c5581d66dd420517c6b (127.30.185.131:35235): tablet 467e8dbe57474750b5f015d5ec0ada09 (table test-table [id=ad1258cd196e446dae2f91b297ae3775]) successfully deleted
I20260811 18:40:55.672904 6462 catalog_manager.cc:4841] TS 839cadbf5ed9449c9440d310dcb7bd5a (127.30.185.129:43809): tablet 467e8dbe57474750b5f015d5ec0ada09 (table test-table [id=ad1258cd196e446dae2f91b297ae3775]) successfully deleted
I20260811 18:40:55.675076 7466 tablet_replica.cc:330] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7: stopping tablet replica
Aug 11 18:40:55 dist-test-slave-z8x9 krb5kdc[6408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.129@KRBTEST.COM
I20260811 18:40:55.675576 7466 raft_consensus.cc:2229] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:40:55.676368 7466 raft_consensus.cc:2258] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:40:55.678329 7466 ts_tablet_manager.cc:1905] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:40:55.684125 7415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:55.648783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33058 (local address 127.30.185.129:43809)
0811 18:40:55.648978 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:55.648984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:55.663374 (+ 14390us) server_negotiation.cc:407] Connection header received
0811 18:40:55.663862 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:55.663868 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:55.663961 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:55.664032 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:55.665935 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.667133 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.669257 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.669921 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.670060 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:55.676342 (+ 6282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:55.676357 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:55.676377 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:55.676431 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:55.680690 (+ 4259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:55.681629 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:55.681639 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:55.681643 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:55.681855 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:55.682325 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:55.682331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:55.682335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:55.682814 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:55.683190 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:55.683448 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 18:40:55.683711 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 18:40:55.684605 7466 ts_tablet_manager.cc:1918] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:40:55.684784 7466 log.cc:1196] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/467e8dbe57474750b5f015d5ec0ada09
I20260811 18:40:55.685017 6872 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:55.685487 7466 ts_tablet_manager.cc:1939] T 467e8dbe57474750b5f015d5ec0ada09 P 5941907cd0ed41f3955d8eeccfab19f7: Deleting consensus metadata
I20260811 18:40:55.687577 6457 catalog_manager.cc:4841] TS 5941907cd0ed41f3955d8eeccfab19f7 (127.30.185.130:39197): tablet 467e8dbe57474750b5f015d5ec0ada09 (table test-table [id=ad1258cd196e446dae2f91b297ae3775]) successfully deleted
I20260811 18:40:55.798543 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 6614
I20260811 18:40:55.830641 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 6891
Aug 11 18:40:55 dist-test-slave-z8x9 krb5kdc[6408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.130@KRBTEST.COM
I20260811 18:40:55.848675 7416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:55.831594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43042 (local address 127.30.185.130:39197)
0811 18:40:55.831823 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:55.831830 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:55.831856 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:40:55.831974 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:55.831981 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:55.832074 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:55.832193 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:55.833534 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.834791 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.836020 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:55.836712 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:55.836861 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:55.841141 (+ 4280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:55.841159 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:55.841199 (+ 40us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:55.841252 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:55.845511 (+ 4259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:55.846294 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:55.846305 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:55.846309 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:55.846484 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:55.846907 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:55.846912 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:55.846916 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:55.847389 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:55.847761 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:55.847972 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 18:40:55.848226 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 18:40:55.849606 7115 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:55.970274 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 6891
I20260811 18:40:55.996985 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 7138
Aug 11 18:40:56 dist-test-slave-z8x9 krb5kdc[6408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.131@KRBTEST.COM
I20260811 18:40:56.012403 7412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:55.997901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41004 (local address 127.30.185.131:35235)
0811 18:40:55.998087 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:55.998091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:55.998110 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:40:55.998179 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:55.998183 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:55.998245 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:55.998346 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:55.999610 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:56.000356 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:56.001737 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:56.002214 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:56.002318 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:56.006979 (+ 4661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:56.006996 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:56.007013 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:40:56.007051 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:56.009973 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:56.010600 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:56.010608 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:56.010610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:56.010745 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:40:56.011083 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:40:56.011086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:40:56.011088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:40:56.011450 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:56.011670 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:56.011948 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 18:40:56.012112 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 18:40:56.013103 7374 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:56.103360 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 7138
I20260811 18:40:56.130373 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 6432
I20260811 18:40:56.131551 6543 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:56.243942 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 6432
2026-08-11T18:40:56Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4934 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T18:40:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:40:56Z Disabled control of system clock
I20260811 18:40:56.334170 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:40417
--webserver_interface=127.30.185.190
--webserver_port=0
--builtin_ntp_servers=127.30.185.148:46327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:40417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 18:40:56.675028 7480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:56.675513 7480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:56.675683 7480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:56.683084 7480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:40:56.683189 7480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:56.683228 7480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:56.683286 7480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:40:56.683336 7480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:40:56.692629 7480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:40417
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:40417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:56.694005 7480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:56.695772 7480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:56.708076 7487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:56.708132 7490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:56.708267 7494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:56.708776 7480 server_base.cc:1034] running on GCE node
I20260811 18:40:56.709702 7480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:56.711100 7480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:56.712312 7480 hybrid_clock.cc:648] HybridClock initialized: now 1786473656712252 us; error 50 us; skew 500 ppm
I20260811 18:40:56.712674 7480 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:40:56.715013 7480 webserver.cc:460] Webserver started at http://127.30.185.190:45759/ using document root <none> and password file <none>
I20260811 18:40:56.715732 7480 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:56.715852 7480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:56.716214 7480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:56.719187 7480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "b1a3680778544032b10f031e8083dfb6"
format_stamp: "Formatted at 2026-08-11 18:40:56 on dist-test-slave-z8x9"
server_key: "991b685bcfe07992bbac1d6dab8a00be"
server_key_iv: "c16f4ad489b7c56ccb7261bff6c05819"
server_key_version: "encryptionkey@0"
I20260811 18:40:56.720007 7480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "b1a3680778544032b10f031e8083dfb6"
format_stamp: "Formatted at 2026-08-11 18:40:56 on dist-test-slave-z8x9"
server_key: "991b685bcfe07992bbac1d6dab8a00be"
server_key_iv: "c16f4ad489b7c56ccb7261bff6c05819"
server_key_version: "encryptionkey@0"
I20260811 18:40:56.725396 7480 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 18:40:56.728786 7506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:56.730152 7480 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:40:56.730366 7480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "b1a3680778544032b10f031e8083dfb6"
format_stamp: "Formatted at 2026-08-11 18:40:56 on dist-test-slave-z8x9"
server_key: "991b685bcfe07992bbac1d6dab8a00be"
server_key_iv: "c16f4ad489b7c56ccb7261bff6c05819"
server_key_version: "encryptionkey@0"
I20260811 18:40:56.730607 7480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:56.736407 7480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:56.737272 7480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:56.737581 7480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:56.756480 7480 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:40417
I20260811 18:40:56.756515 7608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:40417 every 8 connection(s)
I20260811 18:40:56.758008 7480 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:40:56.761777 7609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:56.767416 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 7480
I20260811 18:40:56.767638 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:40:56.768306 31462 external_mini_cluster.cc:1428] Setting key b3314271e5ca53b89186374781a02a94
I20260811 18:40:56.772845 7609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6: Bootstrap starting.
I20260811 18:40:56.776594 7609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:56.777045 7616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:56.770521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41380 (local address 127.30.185.190:40417)
0811 18:40:56.771583 (+ 1062us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:56.771602 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:56.771645 (+ 43us) server_negotiation.cc:407] Connection header received
0811 18:40:56.773420 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:56.773444 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:56.773828 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:56.774140 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:56.775380 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:56.775386 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:56.775398 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:40:56.775412 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:56.775481 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:40:56.775497 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:56.775658 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:56.775799 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 18:40:56.776045 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":378,"threads_started":1}
I20260811 18:40:56.778012 7609 log.cc:824] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:56.781368 7609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6: No bootstrap required, opened a new log
I20260811 18:40:56.788515 7609 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a3680778544032b10f031e8083dfb6" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40417 } }
I20260811 18:40:56.788857 7609 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:56.788931 7609 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1a3680778544032b10f031e8083dfb6, State: Initialized, Role: FOLLOWER
I20260811 18:40:56.789506 7609 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a3680778544032b10f031e8083dfb6" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40417 } }
I20260811 18:40:56.789714 7609 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:40:56.789887 7609 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:40:56.790038 7609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:56.792542 7609 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a3680778544032b10f031e8083dfb6" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40417 } }
I20260811 18:40:56.792999 7609 leader_election.cc:302] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1a3680778544032b10f031e8083dfb6; no voters:
I20260811 18:40:56.793589 7609 leader_election.cc:288] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:40:56.793746 7619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:40:56.794481 7619 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6 [term 1 LEADER]: Becoming Leader. State: Replica: b1a3680778544032b10f031e8083dfb6, State: Running, Role: LEADER
I20260811 18:40:56.795055 7619 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a3680778544032b10f031e8083dfb6" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40417 } }
I20260811 18:40:56.795711 7609 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:40:56.798039 7623 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1a3680778544032b10f031e8083dfb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a3680778544032b10f031e8083dfb6" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40417 } } }
I20260811 18:40:56.798605 7623 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6 [sys.catalog]: This master's current role is: LEADER
I20260811 18:40:56.798188 7624 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1a3680778544032b10f031e8083dfb6. Latest consensus state: current_term: 1 leader_uuid: "b1a3680778544032b10f031e8083dfb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a3680778544032b10f031e8083dfb6" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40417 } } }
I20260811 18:40:56.798926 7624 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6 [sys.catalog]: This master's current role is: LEADER
I20260811 18:40:56.800562 7637 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:40:56.805943 7637 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:40:56.815280 7637 catalog_manager.cc:1347] Generated new cluster ID: 3710ed8611ad4699b326d963b5cb591c
I20260811 18:40:56.815438 7637 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:40:56.833338 7637 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:40:56.833873 7637 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:40:56.843568 7637 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6: Generated new TSK 0
I20260811 18:40:56.844693 7637 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:40:56.856396 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:40417
--builtin_ntp_servers=127.30.185.148:46327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 18:40:57.199368 7652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:57.199678 7652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:57.199836 7652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:57.207454 7652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:57.207571 7652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:57.207764 7652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:40:57.217139 7652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:40417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:57.218588 7652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:57.220494 7652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:57.234400 7664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:57.234421 7665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:57.234452 7667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:57.235293 7652 server_base.cc:1034] running on GCE node
I20260811 18:40:57.235926 7652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:57.236817 7652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:57.238022 7652 hybrid_clock.cc:648] HybridClock initialized: now 1786473657237965 us; error 56 us; skew 500 ppm
I20260811 18:40:57.238363 7652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:40:57.240664 7652 webserver.cc:460] Webserver started at http://127.30.185.129:36441/ using document root <none> and password file <none>
I20260811 18:40:57.241483 7652 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:57.241575 7652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:57.241885 7652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:57.244637 7652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "4cc932c48c7f4d25b4dedea221b9c0e9"
format_stamp: "Formatted at 2026-08-11 18:40:57 on dist-test-slave-z8x9"
server_key: "c2b36a8672e5f762e121bc8e932cfa45"
server_key_iv: "8ae26afefc36153b860dbc63ed512173"
server_key_version: "encryptionkey@0"
I20260811 18:40:57.245514 7652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "4cc932c48c7f4d25b4dedea221b9c0e9"
format_stamp: "Formatted at 2026-08-11 18:40:57 on dist-test-slave-z8x9"
server_key: "c2b36a8672e5f762e121bc8e932cfa45"
server_key_iv: "8ae26afefc36153b860dbc63ed512173"
server_key_version: "encryptionkey@0"
I20260811 18:40:57.250607 7652 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 18:40:57.254043 7684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:57.255366 7652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:40:57.255553 7652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "4cc932c48c7f4d25b4dedea221b9c0e9"
format_stamp: "Formatted at 2026-08-11 18:40:57 on dist-test-slave-z8x9"
server_key: "c2b36a8672e5f762e121bc8e932cfa45"
server_key_iv: "8ae26afefc36153b860dbc63ed512173"
server_key_version: "encryptionkey@0"
I20260811 18:40:57.255777 7652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:57.261551 7652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:57.262372 7652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:57.262648 7652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:57.263552 7652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:40:57.264942 7652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:57.265071 7652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:57.265225 7652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:57.265336 7652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 18:40:57.299320 7652 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:42567
I20260811 18:40:57.299412 7894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:42567 every 8 connection(s)
I20260811 18:40:57.300879 7652 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:40:57.303256 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 7652
I20260811 18:40:57.303499 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:40:57.304322 31462 external_mini_cluster.cc:1428] Setting key e89940ac58cfdd48cb0b96a4b906d06f
I20260811 18:40:57.307365 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:40417
--builtin_ntp_servers=127.30.185.148:46327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 18:40:57.310098 7900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:57.303598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:39229 (local address 127.30.185.190:40417)
0811 18:40:57.304103 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:57.304110 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:57.305572 (+ 1462us) server_negotiation.cc:407] Connection header received
0811 18:40:57.307363 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:57.307369 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:57.307439 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:57.307569 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:57.309387 (+ 1818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:57.309392 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:57.309395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:40:57.309412 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:57.309461 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:40:57.309473 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:57.309574 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:57.309823 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 18:40:57.309879 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":244,"threads_started":1}
I20260811 18:40:57.311595 7901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:57.304270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:40417 (local address 127.30.185.129:39229)
0811 18:40:57.305414 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:57.305468 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:57.307078 (+ 1610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:57.308026 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:57.308040 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:57.308172 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:57.308969 (+ 797us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:40:57.308976 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:57.309035 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:40:57.309189 (+ 154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:57.309580 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:57.309593 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:57.309605 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 18:40:57.309903 (+ 298us) client_negotiation.cc:240] Negotiation successful
0811 18:40:57.310175 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":375,"threads_started":1}
I20260811 18:40:57.313154 7895 heartbeater.cc:347] Connected to a master server at 127.30.185.190:40417
I20260811 18:40:57.313524 7895 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:57.314251 7895 heartbeater.cc:511] Master 127.30.185.190:40417 requested a full tablet report, sending...
I20260811 18:40:57.316478 7540 ts_manager.cc:194] Registered new tserver with Master: 4cc932c48c7f4d25b4dedea221b9c0e9 (127.30.185.129:42567)
W20260811 18:40:57.642083 7904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:57.642369 7904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:57.642534 7904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:57.649894 7904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:57.650008 7904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:57.650192 7904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:40:57.659567 7904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:40417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:57.661018 7904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:57.662889 7904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:57.676555 7925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:57.676676 7931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:57.676708 7926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:57.677865 7904 server_base.cc:1034] running on GCE node
I20260811 18:40:57.678624 7904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:57.679646 7904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:57.680899 7904 hybrid_clock.cc:648] HybridClock initialized: now 1786473657680843 us; error 50 us; skew 500 ppm
I20260811 18:40:57.681291 7904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:40:57.683799 7904 webserver.cc:460] Webserver started at http://127.30.185.130:34349/ using document root <none> and password file <none>
I20260811 18:40:57.684535 7904 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:57.684679 7904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:57.685065 7904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:57.688019 7904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "92c03084bde24b1dac69a45c2eaab5d5"
format_stamp: "Formatted at 2026-08-11 18:40:57 on dist-test-slave-z8x9"
server_key: "9db308e3103061601757e928697bd525"
server_key_iv: "e1d8adeabf838d364b440135954ed010"
server_key_version: "encryptionkey@0"
I20260811 18:40:57.688874 7904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "92c03084bde24b1dac69a45c2eaab5d5"
format_stamp: "Formatted at 2026-08-11 18:40:57 on dist-test-slave-z8x9"
server_key: "9db308e3103061601757e928697bd525"
server_key_iv: "e1d8adeabf838d364b440135954ed010"
server_key_version: "encryptionkey@0"
I20260811 18:40:57.694134 7904 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 18:40:57.697793 7937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:57.699270 7904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:40:57.699468 7904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "92c03084bde24b1dac69a45c2eaab5d5"
format_stamp: "Formatted at 2026-08-11 18:40:57 on dist-test-slave-z8x9"
server_key: "9db308e3103061601757e928697bd525"
server_key_iv: "e1d8adeabf838d364b440135954ed010"
server_key_version: "encryptionkey@0"
I20260811 18:40:57.699714 7904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:57.706215 7904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:57.707053 7904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:57.707342 7904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:57.708316 7904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:40:57.709869 7904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:57.710000 7904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:57.710140 7904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:57.710235 7904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:57.744464 7904 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:45727
I20260811 18:40:57.744513 8190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:45727 every 8 connection(s)
I20260811 18:40:57.745898 7904 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:40:57.753664 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 7904
I20260811 18:40:57.753929 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:40:57.754390 31462 external_mini_cluster.cc:1428] Setting key b79922c93a1a4b4a3d7dc3024351ff0f
I20260811 18:40:57.754663 7900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:57.748700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:45609 (local address 127.30.185.190:40417)
0811 18:40:57.748949 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:57.748962 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:57.750602 (+ 1640us) server_negotiation.cc:407] Connection header received
0811 18:40:57.752252 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:57.752257 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:57.752311 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:57.752403 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:57.753879 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:57.753886 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:57.753891 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:40:57.753916 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:57.753950 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:40:57.753961 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:57.754100 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:57.754301 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 18:40:57.754378 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 18:40:57.755901 8199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:57.749454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:40417 (local address 127.30.185.130:45609)
0811 18:40:57.750438 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:57.750482 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:57.752002 (+ 1520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:57.752607 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:57.752616 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:57.752705 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:57.753527 (+ 822us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:40:57.753533 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:57.753586 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:40:57.753687 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:57.754073 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:57.754086 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:57.754093 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:40:57.754398 (+ 305us) client_negotiation.cc:240] Negotiation successful
0811 18:40:57.754659 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":291,"threads_started":1}
I20260811 18:40:57.757400 8195 heartbeater.cc:347] Connected to a master server at 127.30.185.190:40417
I20260811 18:40:57.757759 8195 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:57.757972 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:40417
--builtin_ntp_servers=127.30.185.148:46327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 18:40:57.758517 8195 heartbeater.cc:511] Master 127.30.185.190:40417 requested a full tablet report, sending...
I20260811 18:40:57.760216 7540 ts_manager.cc:194] Registered new tserver with Master: 92c03084bde24b1dac69a45c2eaab5d5 (127.30.185.130:45727)
W20260811 18:40:58.116822 8201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:58.117134 8201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:58.117298 8201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:58.124593 8201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:58.124691 8201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:58.124800 8201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:40:58.134109 8201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:40417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:58.135458 8201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:58.137197 8201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:58.150666 8209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:58.150755 8213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:58.150698 8210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:58.151583 8201 server_base.cc:1034] running on GCE node
I20260811 18:40:58.152299 8201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:58.153355 8201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:58.154584 8201 hybrid_clock.cc:648] HybridClock initialized: now 1786473658154516 us; error 53 us; skew 500 ppm
I20260811 18:40:58.154946 8201 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:40:58.157437 8201 webserver.cc:460] Webserver started at http://127.30.185.131:41891/ using document root <none> and password file <none>
I20260811 18:40:58.158336 8201 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:58.158485 8201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:58.158933 8201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:58.162269 8201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "af4de57f47074712be04fd152893a2c0"
format_stamp: "Formatted at 2026-08-11 18:40:58 on dist-test-slave-z8x9"
server_key: "a0ac6b4fd4d0a1eca251cff6c445f55a"
server_key_iv: "d9423c67249a385468e8c6e6bfbad5e8"
server_key_version: "encryptionkey@0"
I20260811 18:40:58.163059 8201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "af4de57f47074712be04fd152893a2c0"
format_stamp: "Formatted at 2026-08-11 18:40:58 on dist-test-slave-z8x9"
server_key: "a0ac6b4fd4d0a1eca251cff6c445f55a"
server_key_iv: "d9423c67249a385468e8c6e6bfbad5e8"
server_key_version: "encryptionkey@0"
I20260811 18:40:58.168402 8201 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 18:40:58.172048 8221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:58.173496 8201 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:40:58.173673 8201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "af4de57f47074712be04fd152893a2c0"
format_stamp: "Formatted at 2026-08-11 18:40:58 on dist-test-slave-z8x9"
server_key: "a0ac6b4fd4d0a1eca251cff6c445f55a"
server_key_iv: "d9423c67249a385468e8c6e6bfbad5e8"
server_key_version: "encryptionkey@0"
I20260811 18:40:58.173884 8201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:58.180238 8201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:58.181299 8201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:58.181643 8201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:58.182626 8201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:40:58.183998 8201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:40:58.184087 8201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:58.184166 8201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:40:58.184223 8201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:58.218905 8201 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:42529
I20260811 18:40:58.218961 8441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:42529 every 8 connection(s)
I20260811 18:40:58.220317 8201 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:40:58.225447 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 8201
I20260811 18:40:58.225654 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:40:58.226089 31462 external_mini_cluster.cc:1428] Setting key 8a864165fefa8bc6887be5dcee6fdf70
I20260811 18:40:58.228937 7900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.223105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:36381 (local address 127.30.185.190:40417)
0811 18:40:58.223290 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:58.223295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:58.224636 (+ 1341us) server_negotiation.cc:407] Connection header received
0811 18:40:58.226569 (+ 1933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:58.226575 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:58.226659 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:58.226778 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:58.228165 (+ 1387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:58.228171 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:58.228176 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:40:58.228195 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:58.228237 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:40:58.228249 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:58.228384 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:58.228592 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 18:40:58.228662 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:40:58.230051 8444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.223548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:40417 (local address 127.30.185.131:36381)
0811 18:40:58.224466 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:58.224503 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:58.226277 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:58.227016 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:58.227024 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:58.227115 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:58.227855 (+ 740us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:40:58.227858 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:58.227897 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:40:58.227984 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:58.228355 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:58.228368 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:58.228376 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 18:40:58.228679 (+ 303us) client_negotiation.cc:240] Negotiation successful
0811 18:40:58.228934 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":282,"threads_started":1}
I20260811 18:40:58.231412 8442 heartbeater.cc:347] Connected to a master server at 127.30.185.190:40417
I20260811 18:40:58.231750 8442 heartbeater.cc:464] Registering TS with master...
I20260811 18:40:58.232327 8442 heartbeater.cc:511] Master 127.30.185.190:40417 requested a full tablet report, sending...
I20260811 18:40:58.233723 7540 ts_manager.cc:194] Registered new tserver with Master: af4de57f47074712be04fd152893a2c0 (127.30.185.131:42529)
I20260811 18:40:58.241580 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:40:58.248821 7900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.246153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41390 (local address 127.30.185.190:40417)
0811 18:40:58.246353 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:58.246358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:58.246895 (+ 537us) server_negotiation.cc:407] Connection header received
0811 18:40:58.247030 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:58.247034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:58.247091 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:58.247174 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:58.248161 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:58.248165 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:58.248168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:40:58.248182 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:58.248213 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:40:58.248224 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:58.248327 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:58.248444 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 18:40:58.248495 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 18:40:58.253026 7540 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:40:58.260159 7540 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:40:58.279838 8468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.273499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42674 (local address 127.30.185.130:45727)
0811 18:40:58.274476 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:58.274489 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:58.275196 (+ 707us) server_negotiation.cc:407] Connection header received
0811 18:40:58.275474 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:58.275486 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:58.275799 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:58.275964 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:58.278741 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:58.278751 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:58.278766 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:40:58.278792 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:58.278874 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:40:58.278903 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:58.279018 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:58.279337 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 18:40:58.279456 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":771,"thread_start_us":633,"threads_started":1}
I20260811 18:40:58.280946 8467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.272838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:45727 (local address 127.0.0.1:42674)
0811 18:40:58.275063 (+ 2225us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:58.275086 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:58.275296 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:58.277272 (+ 1976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:58.277284 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:58.277398 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:58.278409 (+ 1011us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:40:58.278413 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:58.278448 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:40:58.278540 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:58.279124 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:58.279135 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:58.279142 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:40:58.280371 (+ 1229us) client_negotiation.cc:240] Negotiation successful
0811 18:40:58.280463 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2045,"thread_start_us":1832,"threads_started":1}
I20260811 18:40:58.282054 8465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.272443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:42567 (local address 127.0.0.1:49214)
0811 18:40:58.273653 (+ 1210us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:58.273705 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:58.273883 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:58.279469 (+ 5586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:58.279473 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:58.279495 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:58.280171 (+ 676us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:40:58.280174 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:58.280189 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:40:58.280232 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:58.281390 (+ 1158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:58.281397 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:58.281405 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 18:40:58.281539 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 18:40:58.281581 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":880,"thread_start_us":643,"threads_started":1}
I20260811 18:40:58.282544 8470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.274684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54920 (local address 127.30.185.131:42529)
0811 18:40:58.278753 (+ 4069us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:58.278761 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:58.279374 (+ 613us) server_negotiation.cc:407] Connection header received
0811 18:40:58.279454 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:58.279462 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:58.279719 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:58.279883 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:58.281532 (+ 1649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:58.281541 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:58.281555 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:40:58.281575 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:58.281631 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:40:58.281651 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:58.281829 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:58.282022 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 18:40:58.282116 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3882,"thread_start_us":3717,"threads_started":1}
I20260811 18:40:58.283010 8466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.272650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49214 (local address 127.30.185.129:42567)
0811 18:40:58.278919 (+ 6269us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:58.278928 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:58.278959 (+ 31us) server_negotiation.cc:407] Connection header received
0811 18:40:58.279030 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:58.279042 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:58.279269 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:58.279908 (+ 639us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:58.281097 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:58.281105 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:58.281119 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:40:58.281137 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:58.281224 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:40:58.281256 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:58.281409 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:58.282179 (+ 770us) server_negotiation.cc:299] Negotiation successful
0811 18:40:58.282667 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5544,"thread_start_us":5394,"threads_started":1}
I20260811 18:40:58.283491 8469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.274033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:42529 (local address 127.0.0.1:54920)
0811 18:40:58.279108 (+ 5075us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:58.279125 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:58.279230 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:58.280396 (+ 1166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:58.280403 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:58.280432 (+ 29us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:58.281243 (+ 811us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:40:58.281247 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:58.281265 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:40:58.281343 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:58.281788 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:58.281800 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:58.281805 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:40:58.281979 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 18:40:58.282036 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4952,"thread_start_us":4813,"threads_started":1}
I20260811 18:40:58.285293 8039 tablet_service.cc:1467] Processing CreateTablet for tablet 50678216067e4628ab4e366402128dfb (DEFAULT_TABLE table=test-table [id=ecac49cbf075476da280181348f5aea9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:58.286732 8039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50678216067e4628ab4e366402128dfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:58.286458 7771 tablet_service.cc:1467] Processing CreateTablet for tablet 50678216067e4628ab4e366402128dfb (DEFAULT_TABLE table=test-table [id=ecac49cbf075476da280181348f5aea9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:58.286937 8314 tablet_service.cc:1467] Processing CreateTablet for tablet 50678216067e4628ab4e366402128dfb (DEFAULT_TABLE table=test-table [id=ecac49cbf075476da280181348f5aea9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:40:58.287833 7771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50678216067e4628ab4e366402128dfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:58.288327 8314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50678216067e4628ab4e366402128dfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:58.309449 8478 tablet_bootstrap.cc:492] T 50678216067e4628ab4e366402128dfb P af4de57f47074712be04fd152893a2c0: Bootstrap starting.
I20260811 18:40:58.312763 8475 tablet_bootstrap.cc:492] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9: Bootstrap starting.
I20260811 18:40:58.314337 8473 tablet_bootstrap.cc:492] T 50678216067e4628ab4e366402128dfb P 92c03084bde24b1dac69a45c2eaab5d5: Bootstrap starting.
I20260811 18:40:58.316332 8475 tablet_bootstrap.cc:654] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:58.317685 8475 log.cc:824] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:58.317864 8473 tablet_bootstrap.cc:654] T 50678216067e4628ab4e366402128dfb P 92c03084bde24b1dac69a45c2eaab5d5: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:58.319149 8473 log.cc:824] T 50678216067e4628ab4e366402128dfb P 92c03084bde24b1dac69a45c2eaab5d5: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:58.319303 7895 heartbeater.cc:503] Master 127.30.185.190:40417 was elected leader, sending a full tablet report...
I20260811 18:40:58.320936 8478 tablet_bootstrap.cc:654] T 50678216067e4628ab4e366402128dfb P af4de57f47074712be04fd152893a2c0: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:58.322499 8478 log.cc:824] T 50678216067e4628ab4e366402128dfb P af4de57f47074712be04fd152893a2c0: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:58.322602 8475 tablet_bootstrap.cc:492] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9: No bootstrap required, opened a new log
I20260811 18:40:58.322945 8475 ts_tablet_manager.cc:1397] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260811 18:40:58.323143 8473 tablet_bootstrap.cc:492] T 50678216067e4628ab4e366402128dfb P 92c03084bde24b1dac69a45c2eaab5d5: No bootstrap required, opened a new log
I20260811 18:40:58.323483 8473 ts_tablet_manager.cc:1397] T 50678216067e4628ab4e366402128dfb P 92c03084bde24b1dac69a45c2eaab5d5: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 18:40:58.328042 8478 tablet_bootstrap.cc:492] T 50678216067e4628ab4e366402128dfb P af4de57f47074712be04fd152893a2c0: No bootstrap required, opened a new log
I20260811 18:40:58.328329 8478 ts_tablet_manager.cc:1397] T 50678216067e4628ab4e366402128dfb P af4de57f47074712be04fd152893a2c0: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.000s
I20260811 18:40:58.334056 8473 raft_consensus.cc:348] T 50678216067e4628ab4e366402128dfb P 92c03084bde24b1dac69a45c2eaab5d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc932c48c7f4d25b4dedea221b9c0e9" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 42567 } } peers { permanent_uuid: "92c03084bde24b1dac69a45c2eaab5d5" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 45727 } } peers { permanent_uuid: "af4de57f47074712be04fd152893a2c0" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42529 } }
I20260811 18:40:58.334056 8475 raft_consensus.cc:348] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc932c48c7f4d25b4dedea221b9c0e9" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 42567 } } peers { permanent_uuid: "92c03084bde24b1dac69a45c2eaab5d5" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 45727 } } peers { permanent_uuid: "af4de57f47074712be04fd152893a2c0" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42529 } }
I20260811 18:40:58.334537 8475 raft_consensus.cc:374] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:58.334677 8475 raft_consensus.cc:729] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cc932c48c7f4d25b4dedea221b9c0e9, State: Initialized, Role: FOLLOWER
I20260811 18:40:58.334708 8473 raft_consensus.cc:374] T 50678216067e4628ab4e366402128dfb P 92c03084bde24b1dac69a45c2eaab5d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:58.335031 8473 raft_consensus.cc:729] T 50678216067e4628ab4e366402128dfb P 92c03084bde24b1dac69a45c2eaab5d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92c03084bde24b1dac69a45c2eaab5d5, State: Initialized, Role: FOLLOWER
I20260811 18:40:58.335511 8475 consensus_queue.cc:260] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc932c48c7f4d25b4dedea221b9c0e9" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 42567 } } peers { permanent_uuid: "92c03084bde24b1dac69a45c2eaab5d5" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 45727 } } peers { permanent_uuid: "af4de57f47074712be04fd152893a2c0" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42529 } }
I20260811 18:40:58.335691 8473 consensus_queue.cc:260] T 50678216067e4628ab4e366402128dfb P 92c03084bde24b1dac69a45c2eaab5d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc932c48c7f4d25b4dedea221b9c0e9" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 42567 } } peers { permanent_uuid: "92c03084bde24b1dac69a45c2eaab5d5" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 45727 } } peers { permanent_uuid: "af4de57f47074712be04fd152893a2c0" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42529 } }
I20260811 18:40:58.335851 8478 raft_consensus.cc:348] T 50678216067e4628ab4e366402128dfb P af4de57f47074712be04fd152893a2c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc932c48c7f4d25b4dedea221b9c0e9" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 42567 } } peers { permanent_uuid: "92c03084bde24b1dac69a45c2eaab5d5" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 45727 } } peers { permanent_uuid: "af4de57f47074712be04fd152893a2c0" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42529 } }
I20260811 18:40:58.336388 8478 raft_consensus.cc:374] T 50678216067e4628ab4e366402128dfb P af4de57f47074712be04fd152893a2c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:58.336542 8478 raft_consensus.cc:729] T 50678216067e4628ab4e366402128dfb P af4de57f47074712be04fd152893a2c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af4de57f47074712be04fd152893a2c0, State: Initialized, Role: FOLLOWER
I20260811 18:40:58.337318 8478 consensus_queue.cc:260] T 50678216067e4628ab4e366402128dfb P af4de57f47074712be04fd152893a2c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc932c48c7f4d25b4dedea221b9c0e9" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 42567 } } peers { permanent_uuid: "92c03084bde24b1dac69a45c2eaab5d5" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 45727 } } peers { permanent_uuid: "af4de57f47074712be04fd152893a2c0" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42529 } }
I20260811 18:40:58.337824 8475 ts_tablet_manager.cc:1428] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260811 18:40:58.338873 8195 heartbeater.cc:503] Master 127.30.185.190:40417 was elected leader, sending a full tablet report...
I20260811 18:40:58.340004 8442 heartbeater.cc:503] Master 127.30.185.190:40417 was elected leader, sending a full tablet report...
I20260811 18:40:58.340214 8478 ts_tablet_manager.cc:1428] T 50678216067e4628ab4e366402128dfb P af4de57f47074712be04fd152893a2c0: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260811 18:40:58.340531 8486 raft_consensus.cc:482] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:40:58.340831 8486 raft_consensus.cc:504] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc932c48c7f4d25b4dedea221b9c0e9" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 42567 } } peers { permanent_uuid: "92c03084bde24b1dac69a45c2eaab5d5" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 45727 } } peers { permanent_uuid: "af4de57f47074712be04fd152893a2c0" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42529 } }
I20260811 18:40:58.341934 8473 ts_tablet_manager.cc:1428] T 50678216067e4628ab4e366402128dfb P 92c03084bde24b1dac69a45c2eaab5d5: Time spent starting tablet: real 0.018s user 0.008s sys 0.008s
I20260811 18:40:58.342903 8486 leader_election.cc:288] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92c03084bde24b1dac69a45c2eaab5d5 (127.30.185.130:45727), af4de57f47074712be04fd152893a2c0 (127.30.185.131:42529)
I20260811 18:40:58.347908 8468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.343439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:52963 (local address 127.30.185.130:45727)
0811 18:40:58.343679 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:58.343687 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:58.345258 (+ 1571us) server_negotiation.cc:407] Connection header received
0811 18:40:58.345474 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:58.345481 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:58.345549 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:58.345648 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:58.346960 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:58.346968 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:58.346973 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:40:58.346994 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:58.347038 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:40:58.347057 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:58.347172 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:58.347568 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 18:40:58.347633 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260811 18:40:58.347971 8492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.343469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:45727 (local address 127.30.185.129:52963)
0811 18:40:58.345135 (+ 1666us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:58.345157 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:58.345349 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:58.345820 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:58.345825 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:58.345848 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:58.346688 (+ 840us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:40:58.346691 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:58.346712 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:40:58.346770 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:58.347236 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:58.347240 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:58.347244 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:40:58.347423 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 18:40:58.347483 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1529,"thread_start_us":1098,"threads_started":1}
I20260811 18:40:58.348131 8470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.344354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:40907 (local address 127.30.185.131:42529)
0811 18:40:58.344558 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:58.344564 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:58.346022 (+ 1458us) server_negotiation.cc:407] Connection header received
0811 18:40:58.346140 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:58.346147 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:58.346203 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:58.346306 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:58.347510 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:58.347515 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:58.347520 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:40:58.347536 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:58.347574 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:40:58.347585 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:58.347711 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:58.347802 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 18:40:58.347871 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 18:40:58.348421 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.343623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:42529 (local address 127.30.185.129:40907)
0811 18:40:58.345877 (+ 2254us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:58.345901 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:58.346023 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:58.346501 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:58.346509 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:58.346534 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:40:58.347270 (+ 736us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:40:58.347274 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:40:58.347290 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:40:58.347360 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:40:58.347678 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:40:58.347682 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:40:58.347688 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:40:58.347866 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 18:40:58.347919 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2128,"thread_start_us":1936,"threads_started":1}
I20260811 18:40:58.349028 8086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50678216067e4628ab4e366402128dfb" candidate_uuid: "4cc932c48c7f4d25b4dedea221b9c0e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92c03084bde24b1dac69a45c2eaab5d5" is_pre_election: true
I20260811 18:40:58.349424 8353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50678216067e4628ab4e366402128dfb" candidate_uuid: "4cc932c48c7f4d25b4dedea221b9c0e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af4de57f47074712be04fd152893a2c0" is_pre_election: true
I20260811 18:40:58.349647 8086 raft_consensus.cc:2454] T 50678216067e4628ab4e366402128dfb P 92c03084bde24b1dac69a45c2eaab5d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cc932c48c7f4d25b4dedea221b9c0e9 in term 0.
I20260811 18:40:58.349951 8353 raft_consensus.cc:2454] T 50678216067e4628ab4e366402128dfb P af4de57f47074712be04fd152893a2c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cc932c48c7f4d25b4dedea221b9c0e9 in term 0.
I20260811 18:40:58.350332 7686 leader_election.cc:302] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cc932c48c7f4d25b4dedea221b9c0e9, 92c03084bde24b1dac69a45c2eaab5d5; no voters:
I20260811 18:40:58.350677 8486 raft_consensus.cc:2781] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:40:58.350841 8486 raft_consensus.cc:482] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:40:58.350960 8486 raft_consensus.cc:3037] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:58.353113 8486 raft_consensus.cc:504] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc932c48c7f4d25b4dedea221b9c0e9" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 42567 } } peers { permanent_uuid: "92c03084bde24b1dac69a45c2eaab5d5" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 45727 } } peers { permanent_uuid: "af4de57f47074712be04fd152893a2c0" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42529 } }
I20260811 18:40:58.353820 8486 leader_election.cc:288] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9 [CANDIDATE]: Term 1 election: Requested vote from peers 92c03084bde24b1dac69a45c2eaab5d5 (127.30.185.130:45727), af4de57f47074712be04fd152893a2c0 (127.30.185.131:42529)
I20260811 18:40:58.354264 8086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50678216067e4628ab4e366402128dfb" candidate_uuid: "4cc932c48c7f4d25b4dedea221b9c0e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92c03084bde24b1dac69a45c2eaab5d5"
I20260811 18:40:58.354400 8353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50678216067e4628ab4e366402128dfb" candidate_uuid: "4cc932c48c7f4d25b4dedea221b9c0e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af4de57f47074712be04fd152893a2c0"
I20260811 18:40:58.354507 8086 raft_consensus.cc:3037] T 50678216067e4628ab4e366402128dfb P 92c03084bde24b1dac69a45c2eaab5d5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:58.354612 8353 raft_consensus.cc:3037] T 50678216067e4628ab4e366402128dfb P af4de57f47074712be04fd152893a2c0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:58.358115 8353 raft_consensus.cc:2454] T 50678216067e4628ab4e366402128dfb P af4de57f47074712be04fd152893a2c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cc932c48c7f4d25b4dedea221b9c0e9 in term 1.
I20260811 18:40:58.358115 8086 raft_consensus.cc:2454] T 50678216067e4628ab4e366402128dfb P 92c03084bde24b1dac69a45c2eaab5d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cc932c48c7f4d25b4dedea221b9c0e9 in term 1.
I20260811 18:40:58.358592 7686 leader_election.cc:302] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cc932c48c7f4d25b4dedea221b9c0e9, 92c03084bde24b1dac69a45c2eaab5d5; no voters:
I20260811 18:40:58.358930 8486 raft_consensus.cc:2781] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:40:58.359438 8486 raft_consensus.cc:686] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9 [term 1 LEADER]: Becoming Leader. State: Replica: 4cc932c48c7f4d25b4dedea221b9c0e9, State: Running, Role: LEADER
I20260811 18:40:58.360051 8486 consensus_queue.cc:237] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc932c48c7f4d25b4dedea221b9c0e9" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 42567 } } peers { permanent_uuid: "92c03084bde24b1dac69a45c2eaab5d5" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 45727 } } peers { permanent_uuid: "af4de57f47074712be04fd152893a2c0" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42529 } }
I20260811 18:40:58.365144 7540 catalog_manager.cc:5495] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cc932c48c7f4d25b4dedea221b9c0e9 (127.30.185.129). New cstate: current_term: 1 leader_uuid: "4cc932c48c7f4d25b4dedea221b9c0e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cc932c48c7f4d25b4dedea221b9c0e9" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 42567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92c03084bde24b1dac69a45c2eaab5d5" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 45727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af4de57f47074712be04fd152893a2c0" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20260811 18:40:58.394157 8466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.391813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49230 (local address 127.30.185.129:42567)
0811 18:40:58.392016 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:58.392022 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:58.392125 (+ 103us) server_negotiation.cc:407] Connection header received
0811 18:40:58.392379 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:58.392384 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:58.392466 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:58.392582 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:58.393500 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:58.393505 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:58.393510 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:40:58.393535 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:58.393568 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:40:58.393580 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:58.393702 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:58.393824 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 18:40:58.393886 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:40:58.402662 8353 raft_consensus.cc:1261] T 50678216067e4628ab4e366402128dfb P af4de57f47074712be04fd152893a2c0 [term 1 FOLLOWER]: Refusing update from remote peer 4cc932c48c7f4d25b4dedea221b9c0e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:40:58.402660 8086 raft_consensus.cc:1261] T 50678216067e4628ab4e366402128dfb P 92c03084bde24b1dac69a45c2eaab5d5 [term 1 FOLLOWER]: Refusing update from remote peer 4cc932c48c7f4d25b4dedea221b9c0e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:40:58.403822 8495 consensus_queue.cc:1035] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af4de57f47074712be04fd152893a2c0" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:40:58.404191 8486 consensus_queue.cc:1035] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92c03084bde24b1dac69a45c2eaab5d5" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 45727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:40:58.415771 8498 mvcc.cc:205] Tried to move back new op lower bound from 7317396104807120896 to 7317396104646483968. Current Snapshot: MvccSnapshot[applied={T|T < 7317396104807120896}]
I20260811 18:40:58.431970 7540 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:40:58.432256 7540 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:40:58.436143 7540 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b1a3680778544032b10f031e8083dfb6: Sending DeleteTablet for 3 replicas of tablet 50678216067e4628ab4e366402128dfb
I20260811 18:40:58.437531 7771 tablet_service.cc:1514] Processing DeleteTablet for tablet 50678216067e4628ab4e366402128dfb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:58 UTC) from {username='slave'} at 127.0.0.1:49214
I20260811 18:40:58.437523 8314 tablet_service.cc:1514] Processing DeleteTablet for tablet 50678216067e4628ab4e366402128dfb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:58 UTC) from {username='slave'} at 127.0.0.1:54920
I20260811 18:40:58.437522 8039 tablet_service.cc:1514] Processing DeleteTablet for tablet 50678216067e4628ab4e366402128dfb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:40:58 UTC) from {username='slave'} at 127.0.0.1:42674
I20260811 18:40:58.440310 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 7652
I20260811 18:40:58.450910 8529 tablet_replica.cc:330] T 50678216067e4628ab4e366402128dfb P af4de57f47074712be04fd152893a2c0: stopping tablet replica
I20260811 18:40:58.451581 8529 raft_consensus.cc:2229] T 50678216067e4628ab4e366402128dfb P af4de57f47074712be04fd152893a2c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:40:58.452275 8529 raft_consensus.cc:2258] T 50678216067e4628ab4e366402128dfb P af4de57f47074712be04fd152893a2c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:40:58.453370 8528 tablet_replica.cc:330] T 50678216067e4628ab4e366402128dfb P 92c03084bde24b1dac69a45c2eaab5d5: stopping tablet replica
I20260811 18:40:58.453929 8528 raft_consensus.cc:2229] T 50678216067e4628ab4e366402128dfb P 92c03084bde24b1dac69a45c2eaab5d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:40:58.454543 8528 raft_consensus.cc:2258] T 50678216067e4628ab4e366402128dfb P 92c03084bde24b1dac69a45c2eaab5d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:40:58.456879 8466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.442099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49240 (local address 127.30.185.129:42567)
0811 18:40:58.442328 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:58.442337 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:58.453873 (+ 11536us) server_negotiation.cc:407] Connection header received
0811 18:40:58.454294 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:58.454301 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:58.454386 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:58.454471 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:58.455964 (+ 1493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:58.455971 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:58.455976 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:40:58.455996 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:58.456035 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:40:58.456050 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:58.456152 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:58.456426 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 18:40:58.456496 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 18:40:58.457998 7887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:58.458271 8528 ts_tablet_manager.cc:1905] T 50678216067e4628ab4e366402128dfb P 92c03084bde24b1dac69a45c2eaab5d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:40:58.465302 8528 ts_tablet_manager.cc:1918] T 50678216067e4628ab4e366402128dfb P 92c03084bde24b1dac69a45c2eaab5d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:40:58.465497 8528 log.cc:1196] T 50678216067e4628ab4e366402128dfb P 92c03084bde24b1dac69a45c2eaab5d5: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/50678216067e4628ab4e366402128dfb
I20260811 18:40:58.465968 8528 ts_tablet_manager.cc:1939] T 50678216067e4628ab4e366402128dfb P 92c03084bde24b1dac69a45c2eaab5d5: Deleting consensus metadata
I20260811 18:40:58.466792 8529 ts_tablet_manager.cc:1905] T 50678216067e4628ab4e366402128dfb P af4de57f47074712be04fd152893a2c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:40:58.467662 7508 catalog_manager.cc:4841] TS 92c03084bde24b1dac69a45c2eaab5d5 (127.30.185.130:45727): tablet 50678216067e4628ab4e366402128dfb (table test-table [id=ecac49cbf075476da280181348f5aea9]) successfully deleted
I20260811 18:40:58.474133 8529 ts_tablet_manager.cc:1918] T 50678216067e4628ab4e366402128dfb P af4de57f47074712be04fd152893a2c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:40:58.474335 8529 log.cc:1196] T 50678216067e4628ab4e366402128dfb P af4de57f47074712be04fd152893a2c0: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/50678216067e4628ab4e366402128dfb
I20260811 18:40:58.474828 8529 ts_tablet_manager.cc:1939] T 50678216067e4628ab4e366402128dfb P af4de57f47074712be04fd152893a2c0: Deleting consensus metadata
I20260811 18:40:58.476435 7511 catalog_manager.cc:4841] TS af4de57f47074712be04fd152893a2c0 (127.30.185.131:42529): tablet 50678216067e4628ab4e366402128dfb (table test-table [id=ecac49cbf075476da280181348f5aea9]) successfully deleted
I20260811 18:40:58.555795 8525 tablet_replica.cc:330] T 50678216067e4628ab4e366402128dfb P 4cc932c48c7f4d25b4dedea221b9c0e9: stopping tablet replica
I20260811 18:40:58.556365 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 7652
W20260811 18:40:58.579432 7511 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.185.129:42567 (error 108)
I20260811 18:40:58.580188 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 7904
I20260811 18:40:58.581485 8469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.580910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:42567 (local address 127.0.0.1:49242)
0811 18:40:58.581114 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:58.581301 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:42567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 18:40:58.581764 7511 catalog_manager.cc:4584] TS 4cc932c48c7f4d25b4dedea221b9c0e9 (127.30.185.129:42567): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 50678216067e4628ab4e366402128dfb: Network error: Client connection negotiation failed: client connection to 127.30.185.129:42567: connect: Connection refused (error 111)
I20260811 18:40:58.583405 8468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.581338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42690 (local address 127.30.185.130:45727)
0811 18:40:58.581506 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:58.581510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:58.581526 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:40:58.581603 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:58.581610 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:58.581723 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:58.581846 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:58.582802 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:58.582806 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:58.582811 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:40:58.582826 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:58.582857 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:40:58.582869 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:58.582967 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:58.583061 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 18:40:58.583106 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:40:58.584028 8182 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:58.635583 8469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.634979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:42567 (local address 127.0.0.1:49250)
0811 18:40:58.635240 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:58.635357 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:42567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 18:40:58.636703 8469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.636300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:42567 (local address 127.0.0.1:49258)
0811 18:40:58.636472 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:58.636563 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:42567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 18:40:58.680007 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 7904
I20260811 18:40:58.697432 8469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.696715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:42567 (local address 127.0.0.1:49266)
0811 18:40:58.696968 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:58.697182 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:42567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 18:40:58.698634 8469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.698173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:42567 (local address 127.0.0.1:49276)
0811 18:40:58.698371 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:58.698472 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:42567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 18:40:58.709136 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 8201
I20260811 18:40:58.712138 8470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.710169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54922 (local address 127.30.185.131:42529)
0811 18:40:58.710362 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:58.710367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:58.710387 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:40:58.710497 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:58.710501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:58.710577 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:58.710663 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:58.711521 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:58.711525 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:58.711528 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:40:58.711543 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:58.711575 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:40:58.711586 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:58.711690 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:58.711753 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 18:40:58.711797 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 18:40:58.712823 8432 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:58.779348 8469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.778846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:42567 (local address 127.0.0.1:49284)
0811 18:40:58.779073 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:58.779176 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:42567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 18:40:58.780573 8469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:58.780183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:42567 (local address 127.0.0.1:49300)
0811 18:40:58.780388 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:58.780459 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:42567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260811 18:40:58.803277 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 8201
I20260811 18:40:58.828146 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 7480
I20260811 18:40:58.829576 7600 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:58.937865 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 7480
2026-08-11T18:40:58Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2694 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 18:40:58.993605 31462 mini_oidc.cc:152] Starting JWKS server
I20260811 18:40:58.996769 31462 webserver.cc:460] Webserver started at https://127.0.0.1:37361/ using document root <none> and password file <none>
I20260811 18:40:58.997252 31462 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 18:40:58.997993 31462 webserver.cc:460] Webserver started at http://127.0.0.1:35869/ using document root <none> and password file <none>
2026-08-11T18:40:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:40:59Z Disabled control of system clock
I20260811 18:40:59.039136 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:44929
--webserver_interface=127.30.185.190
--webserver_port=0
--builtin_ntp_servers=127.30.185.148:33299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:44929
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37361/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786473610573616-31462-0/testchainca.cert with env {}
W20260811 18:40:59.376784 8549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:40:59.377210 8549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:40:59.377399 8549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:40:59.384943 8549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:40:59.385118 8549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:40:59.385243 8549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:40:59.385347 8549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:40:59.385411 8549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 18:40:59.385478 8549 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 18:40:59.385521 8549 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37361/jwks.json
I20260811 18:40:59.395908 8549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:33299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:44929
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:44929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37361/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786473610573616-31462-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:40:59.397342 8549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:40:59.398689 8549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:40:59.410773 8559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:59.410738 8564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:40:59.411064 8558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:40:59.411988 8549 server_base.cc:1034] running on GCE node
I20260811 18:40:59.412796 8549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:40:59.414335 8549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:40:59.415582 8549 hybrid_clock.cc:648] HybridClock initialized: now 1786473659415540 us; error 57 us; skew 500 ppm
I20260811 18:40:59.415957 8549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:40:59.422546 8549 webserver.cc:460] Webserver started at http://127.30.185.190:34339/ using document root <none> and password file <none>
I20260811 18:40:59.423290 8549 fs_manager.cc:359] Metadata directory not provided
I20260811 18:40:59.423448 8549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:40:59.423828 8549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:40:59.426860 8549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "1e46c979294545dda7b2ad64801b5993"
format_stamp: "Formatted at 2026-08-11 18:40:59 on dist-test-slave-z8x9"
server_key: "c95fb0a13456390225f59d81451da19e"
server_key_iv: "fe43c5bd33eb27a2facd4e68012ea7c4"
server_key_version: "encryptionkey@0"
I20260811 18:40:59.427665 8549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "1e46c979294545dda7b2ad64801b5993"
format_stamp: "Formatted at 2026-08-11 18:40:59 on dist-test-slave-z8x9"
server_key: "c95fb0a13456390225f59d81451da19e"
server_key_iv: "fe43c5bd33eb27a2facd4e68012ea7c4"
server_key_version: "encryptionkey@0"
I20260811 18:40:59.432966 8549 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 18:40:59.436539 8572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:40:59.437920 8549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:40:59.438123 8549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "1e46c979294545dda7b2ad64801b5993"
format_stamp: "Formatted at 2026-08-11 18:40:59 on dist-test-slave-z8x9"
server_key: "c95fb0a13456390225f59d81451da19e"
server_key_iv: "fe43c5bd33eb27a2facd4e68012ea7c4"
server_key_version: "encryptionkey@0"
I20260811 18:40:59.438352 8549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:40:59.469558 8549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:40:59.470492 8549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:40:59.470860 8549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:40:59.490401 8549 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:44929
I20260811 18:40:59.490438 8669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:44929 every 8 connection(s)
I20260811 18:40:59.491832 8549 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:40:59.495421 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 8549
I20260811 18:40:59.495644 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:40:59.495858 8670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:40:59.496364 31462 external_mini_cluster.cc:1428] Setting key e3759a8b1e7c13280fdfb7ab6f378bb4
I20260811 18:40:59.510774 8670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e46c979294545dda7b2ad64801b5993: Bootstrap starting.
I20260811 18:40:59.513325 8673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:59.498555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41574 (local address 127.30.185.190:44929)
0811 18:40:59.499737 (+ 1182us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:59.499753 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:59.499804 (+ 51us) server_negotiation.cc:407] Connection header received
0811 18:40:59.502735 (+ 2931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:59.502766 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:59.503394 (+ 628us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:59.503868 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:40:59.504391 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:59.506242 (+ 1851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:59.507769 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:59.508852 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:59.509038 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:59.510393 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:40:59.510405 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:40:59.510425 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:40:59.511128 (+ 703us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:40:59.511227 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:40:59.511253 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:40:59.511385 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 18:40:59.511655 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 18:40:59.511975 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":365,"threads_started":1}
I20260811 18:40:59.514797 8670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e46c979294545dda7b2ad64801b5993: Neither blocks nor log segments found. Creating new log.
I20260811 18:40:59.515875 8670 log.cc:824] T 00000000000000000000000000000000 P 1e46c979294545dda7b2ad64801b5993: Log is configured to *not* fsync() on all Append() calls
I20260811 18:40:59.517274 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:40:59.518783 8670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e46c979294545dda7b2ad64801b5993: No bootstrap required, opened a new log
I20260811 18:40:59.527225 8670 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e46c979294545dda7b2ad64801b5993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e46c979294545dda7b2ad64801b5993" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44929 } }
I20260811 18:40:59.527606 8670 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e46c979294545dda7b2ad64801b5993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:40:59.527706 8670 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e46c979294545dda7b2ad64801b5993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e46c979294545dda7b2ad64801b5993, State: Initialized, Role: FOLLOWER
I20260811 18:40:59.528280 8670 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e46c979294545dda7b2ad64801b5993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e46c979294545dda7b2ad64801b5993" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44929 } }
I20260811 18:40:59.528530 8670 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e46c979294545dda7b2ad64801b5993 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:40:59.528689 8670 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e46c979294545dda7b2ad64801b5993 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:40:59.528872 8670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e46c979294545dda7b2ad64801b5993 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:40:59.531119 8670 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e46c979294545dda7b2ad64801b5993 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e46c979294545dda7b2ad64801b5993" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44929 } }
I20260811 18:40:59.531649 8670 leader_election.cc:302] T 00000000000000000000000000000000 P 1e46c979294545dda7b2ad64801b5993 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e46c979294545dda7b2ad64801b5993; no voters:
I20260811 18:40:59.532323 8670 leader_election.cc:288] T 00000000000000000000000000000000 P 1e46c979294545dda7b2ad64801b5993 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:40:59.532418 8677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e46c979294545dda7b2ad64801b5993 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:40:59.533936 8677 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e46c979294545dda7b2ad64801b5993 [term 1 LEADER]: Becoming Leader. State: Replica: 1e46c979294545dda7b2ad64801b5993, State: Running, Role: LEADER
I20260811 18:40:59.534413 8677 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e46c979294545dda7b2ad64801b5993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e46c979294545dda7b2ad64801b5993" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44929 } }
I20260811 18:40:59.534677 8670 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e46c979294545dda7b2ad64801b5993 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:40:59.537631 8684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e46c979294545dda7b2ad64801b5993 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e46c979294545dda7b2ad64801b5993. Latest consensus state: current_term: 1 leader_uuid: "1e46c979294545dda7b2ad64801b5993" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e46c979294545dda7b2ad64801b5993" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44929 } } }
I20260811 18:40:59.538151 8684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e46c979294545dda7b2ad64801b5993 [sys.catalog]: This master's current role is: LEADER
I20260811 18:40:59.538925 8681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e46c979294545dda7b2ad64801b5993 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e46c979294545dda7b2ad64801b5993" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e46c979294545dda7b2ad64801b5993" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44929 } } }
I20260811 18:40:59.539603 8681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e46c979294545dda7b2ad64801b5993 [sys.catalog]: This master's current role is: LEADER
I20260811 18:40:59.546545 8700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:40:59.550876 8700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:40:59.559787 8700 catalog_manager.cc:1347] Generated new cluster ID: 7397639d3ca945b589ce70330678e82b
I20260811 18:40:59.559916 8700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:40:59.573608 8700 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:40:59.575429 8700 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:40:59.584939 8700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e46c979294545dda7b2ad64801b5993: Generated new TSK 0
I20260811 18:40:59.586057 8700 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 18:40:59.632303 8737 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 18:40:59.632579 8737 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 18:40:59.633905 8673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:40:59.611175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41588 (local address 127.30.185.190:44929)
0811 18:40:59.611382 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 18:40:59.611389 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:40:59.611840 (+ 451us) server_negotiation.cc:407] Connection header received
0811 18:40:59.612164 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:40:59.612175 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:40:59.612276 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:40:59.612447 (+ 171us) server_negotiation.cc:226] Negotiated authn=JWT
0811 18:40:59.612996 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:59.614212 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:59.615659 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:40:59.616383 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:40:59.616517 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:59.617089 (+ 572us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 18:40:59.633114 (+ 16025us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37361/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 18:40:59.633554 (+ 440us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41588: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37361/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":77}
W20260811 18:40:59.634042 8673 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41588: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37361/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 18:40:59.635648 8736 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:40:59.611028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:44929 (local address 127.0.0.1:41588)
0811 18:40:59.611705 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0811 18:40:59.611722 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:40:59.611886 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:40:59.612483 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:40:59.612487 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:40:59.612497 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0811 18:40:59.612846 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:59.612855 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:59.614353 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:59.614357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:59.615512 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:40:59.615526 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:40:59.616569 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:40:59.616574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:40:59.616709 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:40:59.616773 (+ 64us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 18:40:59.633465 (+ 16692us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37361/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 18:40:59.633763 (+ 298us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.185.190:44929: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37361/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":352,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.30.185.190:44929: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37361/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 18:40:59.637862 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 8549
I20260811 18:40:59.639142 8660 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:40:59.736038 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 8549
2026-08-11T18:40:59Z chronyd exiting
I20260811 18:40:59.792104 31462 test_util.cc:175] -----------------------------------------------
I20260811 18:40:59.792243 31462 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786473610573616-31462-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (803 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 18:40:59.796182 31462 mini_oidc.cc:152] Starting JWKS server
I20260811 18:40:59.798158 31462 webserver.cc:460] Webserver started at https://127.0.0.1:41519/ using document root <none> and password file <none>
I20260811 18:40:59.798389 31462 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 18:40:59.799017 31462 webserver.cc:460] Webserver started at http://127.0.0.1:34755/ using document root <none> and password file <none>
2026-08-11T18:40:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:40:59Z Disabled control of system clock
I20260811 18:40:59.838696 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:46677
--webserver_interface=127.30.185.190
--webserver_port=0
--builtin_ntp_servers=127.30.185.148:42091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:46677
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41519/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786473610573616-31462-0/testchainca.cert with env {}
W20260811 18:41:00.183856 8746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:00.184266 8746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:00.184440 8746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:00.193287 8746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:41:00.193441 8746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:00.193509 8746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:00.193593 8746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:41:00.193681 8746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 18:41:00.193771 8746 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 18:41:00.193852 8746 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41519/jwks.json
I20260811 18:41:00.204375 8746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:42091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:46677
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:46677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41519/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786473610573616-31462-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:00.205868 8746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:00.207197 8746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:00.219558 8756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:00.219471 8761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:00.219507 8763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:00.220726 8746 server_base.cc:1034] running on GCE node
I20260811 18:41:00.221684 8746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:00.223187 8746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:00.224417 8746 hybrid_clock.cc:648] HybridClock initialized: now 1786473660224333 us; error 74 us; skew 500 ppm
I20260811 18:41:00.224820 8746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:41:00.227334 8746 webserver.cc:460] Webserver started at http://127.30.185.190:44551/ using document root <none> and password file <none>
I20260811 18:41:00.228080 8746 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:00.228214 8746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:00.228571 8746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:00.231635 8746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "017b8c3fc1a9437fa7441422e705b28c"
format_stamp: "Formatted at 2026-08-11 18:41:00 on dist-test-slave-z8x9"
server_key: "dd326b5e750e32011fc5d3f32418b691"
server_key_iv: "5d9df284524e58feaa2b336d7438ad3c"
server_key_version: "encryptionkey@0"
I20260811 18:41:00.232471 8746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "017b8c3fc1a9437fa7441422e705b28c"
format_stamp: "Formatted at 2026-08-11 18:41:00 on dist-test-slave-z8x9"
server_key: "dd326b5e750e32011fc5d3f32418b691"
server_key_iv: "5d9df284524e58feaa2b336d7438ad3c"
server_key_version: "encryptionkey@0"
I20260811 18:41:00.237980 8746 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 18:41:00.241505 8775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:00.242893 8746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:41:00.243095 8746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "017b8c3fc1a9437fa7441422e705b28c"
format_stamp: "Formatted at 2026-08-11 18:41:00 on dist-test-slave-z8x9"
server_key: "dd326b5e750e32011fc5d3f32418b691"
server_key_iv: "5d9df284524e58feaa2b336d7438ad3c"
server_key_version: "encryptionkey@0"
I20260811 18:41:00.243343 8746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:00.267110 8746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:00.268051 8746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:00.268409 8746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:00.287648 8746 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:46677
I20260811 18:41:00.287669 8862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:46677 every 8 connection(s)
I20260811 18:41:00.289072 8746 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:41:00.292781 8867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:00.294363 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 8746
I20260811 18:41:00.294612 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:41:00.295471 31462 external_mini_cluster.cc:1428] Setting key f71841745f24182b35eff9d90e329cbb
I20260811 18:41:00.307673 8867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 017b8c3fc1a9437fa7441422e705b28c: Bootstrap starting.
I20260811 18:41:00.309705 8872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:00.297589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54444 (local address 127.30.185.190:46677)
0811 18:41:00.298685 (+ 1096us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:00.298696 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:00.298750 (+ 54us) server_negotiation.cc:407] Connection header received
0811 18:41:00.300363 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:00.300386 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:00.300872 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:00.301264 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:00.301828 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:00.303086 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:00.304974 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:00.305769 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:00.305904 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:00.307200 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:00.307208 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:00.307222 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:41:00.307820 (+ 598us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:00.307922 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:41:00.307945 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:00.308034 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:00.308274 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 18:41:00.308632 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":446,"threads_started":1}
I20260811 18:41:00.311194 8867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 017b8c3fc1a9437fa7441422e705b28c: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:00.312325 8867 log.cc:824] T 00000000000000000000000000000000 P 017b8c3fc1a9437fa7441422e705b28c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:41:00.313885 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:41:00.315954 8867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 017b8c3fc1a9437fa7441422e705b28c: No bootstrap required, opened a new log
I20260811 18:41:00.324085 8867 raft_consensus.cc:348] T 00000000000000000000000000000000 P 017b8c3fc1a9437fa7441422e705b28c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017b8c3fc1a9437fa7441422e705b28c" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46677 } }
I20260811 18:41:00.324455 8867 raft_consensus.cc:374] T 00000000000000000000000000000000 P 017b8c3fc1a9437fa7441422e705b28c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:00.324539 8867 raft_consensus.cc:729] T 00000000000000000000000000000000 P 017b8c3fc1a9437fa7441422e705b28c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 017b8c3fc1a9437fa7441422e705b28c, State: Initialized, Role: FOLLOWER
I20260811 18:41:00.325119 8867 consensus_queue.cc:260] T 00000000000000000000000000000000 P 017b8c3fc1a9437fa7441422e705b28c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017b8c3fc1a9437fa7441422e705b28c" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46677 } }
I20260811 18:41:00.325431 8867 raft_consensus.cc:388] T 00000000000000000000000000000000 P 017b8c3fc1a9437fa7441422e705b28c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:41:00.325543 8867 raft_consensus.cc:482] T 00000000000000000000000000000000 P 017b8c3fc1a9437fa7441422e705b28c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:41:00.325671 8867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 017b8c3fc1a9437fa7441422e705b28c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:00.328143 8867 raft_consensus.cc:504] T 00000000000000000000000000000000 P 017b8c3fc1a9437fa7441422e705b28c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017b8c3fc1a9437fa7441422e705b28c" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46677 } }
I20260811 18:41:00.328634 8867 leader_election.cc:302] T 00000000000000000000000000000000 P 017b8c3fc1a9437fa7441422e705b28c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 017b8c3fc1a9437fa7441422e705b28c; no voters:
I20260811 18:41:00.329253 8867 leader_election.cc:288] T 00000000000000000000000000000000 P 017b8c3fc1a9437fa7441422e705b28c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:41:00.329406 8875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 017b8c3fc1a9437fa7441422e705b28c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:41:00.330159 8875 raft_consensus.cc:686] T 00000000000000000000000000000000 P 017b8c3fc1a9437fa7441422e705b28c [term 1 LEADER]: Becoming Leader. State: Replica: 017b8c3fc1a9437fa7441422e705b28c, State: Running, Role: LEADER
I20260811 18:41:00.330783 8875 consensus_queue.cc:237] T 00000000000000000000000000000000 P 017b8c3fc1a9437fa7441422e705b28c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017b8c3fc1a9437fa7441422e705b28c" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46677 } }
I20260811 18:41:00.331575 8867 sys_catalog.cc:564] T 00000000000000000000000000000000 P 017b8c3fc1a9437fa7441422e705b28c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:41:00.333956 8877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 017b8c3fc1a9437fa7441422e705b28c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 017b8c3fc1a9437fa7441422e705b28c. Latest consensus state: current_term: 1 leader_uuid: "017b8c3fc1a9437fa7441422e705b28c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017b8c3fc1a9437fa7441422e705b28c" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46677 } } }
I20260811 18:41:00.334414 8877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 017b8c3fc1a9437fa7441422e705b28c [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:00.334820 8876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 017b8c3fc1a9437fa7441422e705b28c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "017b8c3fc1a9437fa7441422e705b28c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "017b8c3fc1a9437fa7441422e705b28c" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46677 } } }
I20260811 18:41:00.335176 8876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 017b8c3fc1a9437fa7441422e705b28c [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:00.336680 8887 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:41:00.342137 8887 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:41:00.351109 8887 catalog_manager.cc:1347] Generated new cluster ID: fe6ed6b9ca244317b50f96197ced099e
I20260811 18:41:00.351213 8887 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:41:00.377562 8887 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:41:00.379438 8887 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:41:00.387537 8887 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 017b8c3fc1a9437fa7441422e705b28c: Generated new TSK 0
I20260811 18:41:00.388590 8887 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 18:41:00.422165 8921 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 18:41:00.422310 8921 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 18:41:00.423633 8872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:00.402873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54448 (local address 127.30.185.190:46677)
0811 18:41:00.403062 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:00.403070 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:00.403444 (+ 374us) server_negotiation.cc:407] Connection header received
0811 18:41:00.403693 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:00.403699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:00.403795 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:00.403985 (+ 190us) server_negotiation.cc:226] Negotiated authn=JWT
0811 18:41:00.404447 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:00.405637 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:00.407003 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:00.407716 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:00.407847 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:00.408357 (+ 510us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 18:41:00.422892 (+ 14535us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 18:41:00.423284 (+ 392us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54448: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 18:41:00.423655 8920 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:00.402694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46677 (local address 127.0.0.1:54448)
0811 18:41:00.403329 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:00.403348 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:00.403499 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:00.403996 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:00.403999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:00.404009 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0811 18:41:00.404277 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:00.404285 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:00.405780 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:00.405784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:00.406860 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:00.406871 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:00.407889 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:00.407894 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:00.408034 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:00.408068 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 18:41:00.423159 (+ 15091us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 18:41:00.423363 (+ 204us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.185.190:46677: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":339,"threads_started":1}
W20260811 18:41:00.423780 8872 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54448: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260811 18:41:00.425388 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 8746
I20260811 18:41:00.426611 8858 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:00.521495 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 8746
2026-08-11T18:41:00Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (787 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 18:41:00.584077 31462 mini_oidc.cc:152] Starting JWKS server
I20260811 18:41:00.586261 31462 webserver.cc:460] Webserver started at https://127.0.0.1:43963/ using document root <none> and password file <none>
I20260811 18:41:00.586537 31462 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 18:41:00.587256 31462 webserver.cc:460] Webserver started at http://127.0.0.1:44673/ using document root <none> and password file <none>
2026-08-11T18:41:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:41:00Z Disabled control of system clock
I20260811 18:41:00.627040 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:41405
--webserver_interface=127.30.185.190
--webserver_port=0
--builtin_ntp_servers=127.30.185.148:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:41405
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43963/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 18:41:00.969790 8937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:00.970300 8937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:00.970463 8937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:00.978163 8937 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:41:00.978330 8937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:00.978426 8937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:00.978546 8937 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:41:00.978641 8937 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 18:41:00.978737 8937 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 18:41:00.978813 8937 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43963/jwks.json
I20260811 18:41:00.988478 8937 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:41405
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:41405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43963/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:00.990039 8937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:00.991849 8937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:01.004169 8943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:01.004027 8946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:01.004112 8944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:01.005347 8937 server_base.cc:1034] running on GCE node
I20260811 18:41:01.006232 8937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:01.007730 8937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:01.008980 8937 hybrid_clock.cc:648] HybridClock initialized: now 1786473661008941 us; error 49 us; skew 500 ppm
I20260811 18:41:01.009413 8937 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:41:01.011991 8937 webserver.cc:460] Webserver started at http://127.30.185.190:33427/ using document root <none> and password file <none>
I20260811 18:41:01.012715 8937 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:01.012853 8937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:01.013276 8937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:01.016232 8937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "6d09556b821d4b4aa1d76a31c9ad0e85"
format_stamp: "Formatted at 2026-08-11 18:41:01 on dist-test-slave-z8x9"
server_key: "48bf108dba76ee9a6098324100a78671"
server_key_iv: "25e15d84f660bcff155d6c1a3c6b962a"
server_key_version: "encryptionkey@0"
I20260811 18:41:01.017112 8937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "6d09556b821d4b4aa1d76a31c9ad0e85"
format_stamp: "Formatted at 2026-08-11 18:41:01 on dist-test-slave-z8x9"
server_key: "48bf108dba76ee9a6098324100a78671"
server_key_iv: "25e15d84f660bcff155d6c1a3c6b962a"
server_key_version: "encryptionkey@0"
I20260811 18:41:01.022339 8937 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 18:41:01.025872 8959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:01.027173 8937 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:41:01.027387 8937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "6d09556b821d4b4aa1d76a31c9ad0e85"
format_stamp: "Formatted at 2026-08-11 18:41:01 on dist-test-slave-z8x9"
server_key: "48bf108dba76ee9a6098324100a78671"
server_key_iv: "25e15d84f660bcff155d6c1a3c6b962a"
server_key_version: "encryptionkey@0"
I20260811 18:41:01.027621 8937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:01.044616 8937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:01.045591 8937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:01.045923 8937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:01.064860 8937 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:41405
I20260811 18:41:01.064884 9051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:41405 every 8 connection(s)
I20260811 18:41:01.066301 8937 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:41:01.070108 9055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:01.073328 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 8937
I20260811 18:41:01.073539 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:41:01.074373 31462 external_mini_cluster.cc:1428] Setting key 62953aa7905cc4b04ab2186b2a8dac5b
I20260811 18:41:01.082846 9055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d09556b821d4b4aa1d76a31c9ad0e85: Bootstrap starting.
I20260811 18:41:01.086069 9055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d09556b821d4b4aa1d76a31c9ad0e85: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:01.087260 9055 log.cc:824] T 00000000000000000000000000000000 P 6d09556b821d4b4aa1d76a31c9ad0e85: Log is configured to *not* fsync() on all Append() calls
I20260811 18:41:01.088677 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:01.076590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37054 (local address 127.30.185.190:41405)
0811 18:41:01.077641 (+ 1051us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:01.077653 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:01.077694 (+ 41us) server_negotiation.cc:407] Connection header received
0811 18:41:01.079392 (+ 1698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:01.079415 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:01.079893 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:01.080316 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:01.080907 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:01.082243 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:01.083673 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:01.084370 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:01.084569 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:01.085857 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:01.085865 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:01.085879 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:41:01.086523 (+ 644us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:01.086624 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:41:01.086651 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:01.086777 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:01.087024 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 18:41:01.087335 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":357,"threads_started":1}
I20260811 18:41:01.090730 9055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d09556b821d4b4aa1d76a31c9ad0e85: No bootstrap required, opened a new log
I20260811 18:41:01.092782 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:41:01.099352 9055 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d09556b821d4b4aa1d76a31c9ad0e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d09556b821d4b4aa1d76a31c9ad0e85" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41405 } }
I20260811 18:41:01.099833 9055 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d09556b821d4b4aa1d76a31c9ad0e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:01.099963 9055 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d09556b821d4b4aa1d76a31c9ad0e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d09556b821d4b4aa1d76a31c9ad0e85, State: Initialized, Role: FOLLOWER
I20260811 18:41:01.100723 9055 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d09556b821d4b4aa1d76a31c9ad0e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d09556b821d4b4aa1d76a31c9ad0e85" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41405 } }
I20260811 18:41:01.101044 9055 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d09556b821d4b4aa1d76a31c9ad0e85 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:41:01.101228 9055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d09556b821d4b4aa1d76a31c9ad0e85 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:41:01.101446 9055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d09556b821d4b4aa1d76a31c9ad0e85 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:01.103724 9055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d09556b821d4b4aa1d76a31c9ad0e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d09556b821d4b4aa1d76a31c9ad0e85" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41405 } }
I20260811 18:41:01.104329 9055 leader_election.cc:302] T 00000000000000000000000000000000 P 6d09556b821d4b4aa1d76a31c9ad0e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d09556b821d4b4aa1d76a31c9ad0e85; no voters:
I20260811 18:41:01.105007 9055 leader_election.cc:288] T 00000000000000000000000000000000 P 6d09556b821d4b4aa1d76a31c9ad0e85 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:41:01.105218 9065 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d09556b821d4b4aa1d76a31c9ad0e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:41:01.106488 9065 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d09556b821d4b4aa1d76a31c9ad0e85 [term 1 LEADER]: Becoming Leader. State: Replica: 6d09556b821d4b4aa1d76a31c9ad0e85, State: Running, Role: LEADER
I20260811 18:41:01.107080 9065 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d09556b821d4b4aa1d76a31c9ad0e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d09556b821d4b4aa1d76a31c9ad0e85" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41405 } }
I20260811 18:41:01.107498 9055 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d09556b821d4b4aa1d76a31c9ad0e85 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:41:01.110445 9067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d09556b821d4b4aa1d76a31c9ad0e85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d09556b821d4b4aa1d76a31c9ad0e85. Latest consensus state: current_term: 1 leader_uuid: "6d09556b821d4b4aa1d76a31c9ad0e85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d09556b821d4b4aa1d76a31c9ad0e85" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41405 } } }
I20260811 18:41:01.110948 9067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d09556b821d4b4aa1d76a31c9ad0e85 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:01.110471 9066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d09556b821d4b4aa1d76a31c9ad0e85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d09556b821d4b4aa1d76a31c9ad0e85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d09556b821d4b4aa1d76a31c9ad0e85" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41405 } } }
I20260811 18:41:01.111860 9066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d09556b821d4b4aa1d76a31c9ad0e85 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:01.112027 9082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:41:01.117738 9082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:41:01.127028 9082 catalog_manager.cc:1347] Generated new cluster ID: f1b134b7505242c59f39b2ecbde177f0
I20260811 18:41:01.127167 9082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:41:01.141479 9082 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:41:01.142995 9082 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:41:01.153503 9082 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d09556b821d4b4aa1d76a31c9ad0e85: Generated new TSK 0
I20260811 18:41:01.154480 9082 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:41:01.188540 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:01.179412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37062 (local address 127.30.185.190:41405)
0811 18:41:01.179673 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:01.179683 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:01.180243 (+ 560us) server_negotiation.cc:407] Connection header received
0811 18:41:01.180510 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:01.180514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:01.180589 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:01.180732 (+ 143us) server_negotiation.cc:226] Negotiated authn=JWT
0811 18:41:01.181343 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:01.182102 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:01.184208 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:01.184667 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:01.184774 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:01.185453 (+ 679us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 18:41:01.187855 (+ 2402us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43963/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 18:41:01.188210 (+ 355us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37062: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43963/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":83}
W20260811 18:41:01.188687 9062 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37062: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43963/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 18:41:01.191145 9108 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:01.179383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41405 (local address 127.0.0.1:37062)
0811 18:41:01.180081 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:01.180107 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:01.180294 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:01.180786 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:01.180792 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:01.180806 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0811 18:41:01.181141 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:01.181153 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:01.182277 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:01.182283 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:01.184034 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:01.184054 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:01.184880 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:01.184887 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:01.185077 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:01.185128 (+ 51us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 18:41:01.188081 (+ 2953us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43963/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 18:41:01.190661 (+ 2580us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.185.190:41405: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43963/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":547,"mutex_wait_us":2226,"thread_start_us":353,"threads_started":1}
I20260811 18:41:01.192399 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 8937
I20260811 18:41:01.193857 9046 generic_service.cc:196] Checking for leaks (request via RPC)
E20260811 18:41:01.206511 9110 webserver.cc:533] Webserver: sslize failed:
E20260811 18:41:01.206669 9110 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 18:41:01.289453 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 8937
2026-08-11T18:41:01Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (770 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T18:41:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:41:01Z Disabled control of system clock
I20260811 18:41:01.394240 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:33491
--webserver_interface=127.30.185.190
--webserver_port=0
--builtin_ntp_servers=127.30.185.148:32909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:33491
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786473610573616-31462-0/testchainca.cert with env {}
W20260811 18:41:01.728686 9116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:01.729044 9116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:01.729274 9116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:01.736613 9116 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:41:01.737030 9116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:01.737151 9116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:01.737304 9116 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:41:01.737386 9116 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 18:41:01.737483 9116 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 18:41:01.737564 9116 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 18:41:01.747819 9116 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:32909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:33491
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:33491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786473610573616-31462-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:01.749279 9116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:01.750609 9116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:01.762629 9126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:01.762718 9130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:01.762918 9124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:01.763890 9116 server_base.cc:1034] running on GCE node
I20260811 18:41:01.764782 9116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:01.766372 9116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:01.767647 9116 hybrid_clock.cc:648] HybridClock initialized: now 1786473661767588 us; error 55 us; skew 500 ppm
I20260811 18:41:01.767998 9116 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:41:01.770483 9116 webserver.cc:460] Webserver started at http://127.30.185.190:35885/ using document root <none> and password file <none>
I20260811 18:41:01.771188 9116 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:01.771319 9116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:01.771656 9116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:01.774616 9116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "67232d6fa0414625a35d493d413c0b80"
format_stamp: "Formatted at 2026-08-11 18:41:01 on dist-test-slave-z8x9"
server_key: "05f767ee71b9bf0efef9beba94a65cc0"
server_key_iv: "abbaea2b93591c44ee51f68515a3afa0"
server_key_version: "encryptionkey@0"
I20260811 18:41:01.775430 9116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "67232d6fa0414625a35d493d413c0b80"
format_stamp: "Formatted at 2026-08-11 18:41:01 on dist-test-slave-z8x9"
server_key: "05f767ee71b9bf0efef9beba94a65cc0"
server_key_iv: "abbaea2b93591c44ee51f68515a3afa0"
server_key_version: "encryptionkey@0"
I20260811 18:41:01.780678 9116 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 18:41:01.784369 9142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:01.785827 9116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:41:01.786031 9116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "67232d6fa0414625a35d493d413c0b80"
format_stamp: "Formatted at 2026-08-11 18:41:01 on dist-test-slave-z8x9"
server_key: "05f767ee71b9bf0efef9beba94a65cc0"
server_key_iv: "abbaea2b93591c44ee51f68515a3afa0"
server_key_version: "encryptionkey@0"
I20260811 18:41:01.786273 9116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:01.805711 9116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:01.806628 9116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:01.806947 9116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:01.826197 9116 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:33491
I20260811 18:41:01.826233 9235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:33491 every 8 connection(s)
I20260811 18:41:01.827631 9116 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:41:01.829082 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 9116
I20260811 18:41:01.829344 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:41:01.830195 31462 external_mini_cluster.cc:1428] Setting key 2fdd4dc45b939524d4d39490be8c76ea
I20260811 18:41:01.832787 9236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:01.843730 9239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:01.832202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35982 (local address 127.30.185.190:33491)
0811 18:41:01.833331 (+ 1129us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:01.833343 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:01.833388 (+ 45us) server_negotiation.cc:407] Connection header received
0811 18:41:01.835001 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:01.835023 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:01.835493 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:01.835929 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:01.836501 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:01.837964 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:01.839278 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:01.839940 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:01.840133 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:01.841448 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:01.841456 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:01.841472 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:41:01.842049 (+ 577us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:01.842126 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:41:01.842148 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:01.842275 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:01.842504 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 18:41:01.842810 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":351,"threads_started":1}
I20260811 18:41:01.843832 9236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67232d6fa0414625a35d493d413c0b80: Bootstrap starting.
I20260811 18:41:01.847227 9236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67232d6fa0414625a35d493d413c0b80: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:01.847585 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:41:01.848328 9236 log.cc:824] T 00000000000000000000000000000000 P 67232d6fa0414625a35d493d413c0b80: Log is configured to *not* fsync() on all Append() calls
I20260811 18:41:01.851342 9236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67232d6fa0414625a35d493d413c0b80: No bootstrap required, opened a new log
I20260811 18:41:01.859097 9236 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67232d6fa0414625a35d493d413c0b80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67232d6fa0414625a35d493d413c0b80" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33491 } }
I20260811 18:41:01.859503 9236 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67232d6fa0414625a35d493d413c0b80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:01.859618 9236 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67232d6fa0414625a35d493d413c0b80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67232d6fa0414625a35d493d413c0b80, State: Initialized, Role: FOLLOWER
I20260811 18:41:01.860163 9236 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67232d6fa0414625a35d493d413c0b80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67232d6fa0414625a35d493d413c0b80" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33491 } }
I20260811 18:41:01.860610 9236 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67232d6fa0414625a35d493d413c0b80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:41:01.860800 9236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67232d6fa0414625a35d493d413c0b80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:41:01.861003 9236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67232d6fa0414625a35d493d413c0b80 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:01.863368 9236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67232d6fa0414625a35d493d413c0b80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67232d6fa0414625a35d493d413c0b80" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33491 } }
I20260811 18:41:01.863950 9236 leader_election.cc:302] T 00000000000000000000000000000000 P 67232d6fa0414625a35d493d413c0b80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67232d6fa0414625a35d493d413c0b80; no voters:
I20260811 18:41:01.864648 9236 leader_election.cc:288] T 00000000000000000000000000000000 P 67232d6fa0414625a35d493d413c0b80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:41:01.864794 9241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67232d6fa0414625a35d493d413c0b80 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:41:01.865574 9241 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67232d6fa0414625a35d493d413c0b80 [term 1 LEADER]: Becoming Leader. State: Replica: 67232d6fa0414625a35d493d413c0b80, State: Running, Role: LEADER
I20260811 18:41:01.866196 9241 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67232d6fa0414625a35d493d413c0b80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67232d6fa0414625a35d493d413c0b80" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33491 } }
I20260811 18:41:01.867178 9236 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67232d6fa0414625a35d493d413c0b80 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:41:01.869367 9242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67232d6fa0414625a35d493d413c0b80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67232d6fa0414625a35d493d413c0b80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67232d6fa0414625a35d493d413c0b80" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33491 } } }
I20260811 18:41:01.869469 9247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67232d6fa0414625a35d493d413c0b80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67232d6fa0414625a35d493d413c0b80. Latest consensus state: current_term: 1 leader_uuid: "67232d6fa0414625a35d493d413c0b80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67232d6fa0414625a35d493d413c0b80" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33491 } } }
I20260811 18:41:01.869825 9242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67232d6fa0414625a35d493d413c0b80 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:01.869820 9247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67232d6fa0414625a35d493d413c0b80 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:01.873817 9257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:41:01.879738 9257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:41:01.887487 9257 catalog_manager.cc:1347] Generated new cluster ID: f7fd66b2bf034380a2fbc5ef05cebdc9
I20260811 18:41:01.887596 9257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:41:01.898658 9257 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:41:01.899978 9257 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:41:01.908203 9257 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67232d6fa0414625a35d493d413c0b80: Generated new TSK 0
I20260811 18:41:01.909044 9257 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:41:02.835103 9239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:01.927999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35984 (local address 127.30.185.190:33491)
0811 18:41:01.928172 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:01.928177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:01.928583 (+ 406us) server_negotiation.cc:407] Connection header received
0811 18:41:01.928863 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:01.928867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:01.928939 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:01.929083 (+ 144us) server_negotiation.cc:226] Negotiated authn=JWT
0811 18:41:01.929689 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:01.930424 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:01.932600 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:01.933075 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:01.933235 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:01.933813 (+ 578us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 18:41:02.834330 (+900517us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 18:41:02.834767 (+ 437us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35984: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 18:41:02.835248 9239 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35984: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 18:41:02.835652 9294 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:01.927810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:33491 (local address 127.0.0.1:35984)
0811 18:41:01.928464 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:01.928485 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:01.928657 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:01.929133 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:01.929139 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:01.929152 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0811 18:41:01.929514 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:01.929529 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:01.930578 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:01.930584 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:01.932424 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:01.932441 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:01.933283 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:01.933290 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:01.933480 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:01.933528 (+ 48us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 18:41:02.834635 (+901107us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 18:41:02.834959 (+ 324us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.185.190:33491: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":354,"threads_started":1}
I20260811 18:41:02.836737 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 9116
I20260811 18:41:02.838001 9225 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:02.942078 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 9116
2026-08-11T18:41:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1631 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (440 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (428 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:41:03 dist-test-slave-z8x9 krb5kdc[9322](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:41:03 dist-test-slave-z8x9 krb5kdc[9322](info): set up 2 sockets
Aug 11 18:41:03 dist-test-slave-z8x9 krb5kdc[9322](info): commencing operation
krb5kdc: starting...
W20260811 18:41:05.300350 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.444s user 0.002s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:41:05 dist-test-slave-z8x9 krb5kdc[9322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:41:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:41:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:41:05.578648 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:35179
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:37095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:35179
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:41:05.914606 9347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:05.914915 9347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:05.915010 9347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:05.922300 9347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:41:05.922434 9347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:05.922549 9347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:05.922664 9347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:41:05.922753 9347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:41:05.932847 9347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:37095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:35179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:35179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:05.934340 9347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:05.935717 9347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:05.948001 9357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:05.948096 9355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:05.948063 9359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:05.948905 9347 server_base.cc:1034] running on GCE node
I20260811 18:41:05.949786 9347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:05.951287 9347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:05.952610 9347 hybrid_clock.cc:648] HybridClock initialized: now 1786473665952573 us; error 57 us; skew 500 ppm
I20260811 18:41:05.953033 9347 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:05 dist-test-slave-z8x9 krb5kdc[9322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473665, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:05.961293 9347 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:41:05.962970 9347 webserver.cc:460] Webserver started at http://127.30.185.190:37895/ using document root <none> and password file <none>
I20260811 18:41:05.963732 9347 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:05.963912 9347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:05.964277 9347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:05.967409 9347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "ce4f9a3467964a68800cd9b2e2e7536b"
format_stamp: "Formatted at 2026-08-11 18:41:05 on dist-test-slave-z8x9"
server_key: "1d7667f236516f8384ffbf6167aebe43"
server_key_iv: "793b812641e83f2fafd280ac12056beb"
server_key_version: "encryptionkey@0"
I20260811 18:41:05.968227 9347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "ce4f9a3467964a68800cd9b2e2e7536b"
format_stamp: "Formatted at 2026-08-11 18:41:05 on dist-test-slave-z8x9"
server_key: "1d7667f236516f8384ffbf6167aebe43"
server_key_iv: "793b812641e83f2fafd280ac12056beb"
server_key_version: "encryptionkey@0"
I20260811 18:41:05.973830 9347 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 18:41:05.977522 9375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:05.978814 9347 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:41:05.979007 9347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "ce4f9a3467964a68800cd9b2e2e7536b"
format_stamp: "Formatted at 2026-08-11 18:41:05 on dist-test-slave-z8x9"
server_key: "1d7667f236516f8384ffbf6167aebe43"
server_key_iv: "793b812641e83f2fafd280ac12056beb"
server_key_version: "encryptionkey@0"
I20260811 18:41:05.979233 9347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:05.996440 9347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:06.000876 9347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:06.001250 9347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:06.020532 9347 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:35179
I20260811 18:41:06.020566 9480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:35179 every 8 connection(s)
I20260811 18:41:06.022045 9347 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:41:06.024544 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 9347
I20260811 18:41:06.024828 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:41:06.025563 31462 external_mini_cluster.cc:1428] Setting key 375c4dd81c7b45a9aed5954b4d849469
I20260811 18:41:06.026119 9482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:06.041222 9482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce4f9a3467964a68800cd9b2e2e7536b: Bootstrap starting.
Aug 11 18:41:06 dist-test-slave-z8x9 krb5kdc[9322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:41:06.045490 9482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce4f9a3467964a68800cd9b2e2e7536b: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:06.046983 9482 log.cc:824] T 00000000000000000000000000000000 P ce4f9a3467964a68800cd9b2e2e7536b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:41:06.050788 9482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce4f9a3467964a68800cd9b2e2e7536b: No bootstrap required, opened a new log
I20260811 18:41:06.052529 9485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:06.027866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59412 (local address 127.30.185.190:35179)
0811 18:41:06.028733 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:06.028750 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:06.028802 (+ 52us) server_negotiation.cc:407] Connection header received
0811 18:41:06.031322 (+ 2520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:06.031354 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:06.031975 (+ 621us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:06.032465 (+ 490us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:06.033983 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:06.035968 (+ 1985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:06.037323 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:06.038478 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:06.038684 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:06.042837 (+ 4153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:06.042854 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:06.042876 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:06.042943 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:06.047378 (+ 4435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:06.048392 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:06.048399 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:06.048406 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:06.048673 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:06.049070 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:06.049075 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:06.049078 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:06.049860 (+ 782us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:06.050175 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:06.050579 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 18:41:06.051039 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":333,"threads_started":1}
I20260811 18:41:06.060024 9482 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce4f9a3467964a68800cd9b2e2e7536b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4f9a3467964a68800cd9b2e2e7536b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35179 } }
I20260811 18:41:06.060375 9482 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce4f9a3467964a68800cd9b2e2e7536b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:06.060457 9482 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce4f9a3467964a68800cd9b2e2e7536b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce4f9a3467964a68800cd9b2e2e7536b, State: Initialized, Role: FOLLOWER
I20260811 18:41:06.061024 9482 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce4f9a3467964a68800cd9b2e2e7536b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4f9a3467964a68800cd9b2e2e7536b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35179 } }
I20260811 18:41:06.061353 9482 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce4f9a3467964a68800cd9b2e2e7536b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:41:06.061512 9482 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce4f9a3467964a68800cd9b2e2e7536b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:41:06.061745 9482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce4f9a3467964a68800cd9b2e2e7536b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:06.064152 9482 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce4f9a3467964a68800cd9b2e2e7536b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4f9a3467964a68800cd9b2e2e7536b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35179 } }
I20260811 18:41:06.064692 9482 leader_election.cc:302] T 00000000000000000000000000000000 P ce4f9a3467964a68800cd9b2e2e7536b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce4f9a3467964a68800cd9b2e2e7536b; no voters:
I20260811 18:41:06.065454 9482 leader_election.cc:288] T 00000000000000000000000000000000 P ce4f9a3467964a68800cd9b2e2e7536b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:41:06.065587 9489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce4f9a3467964a68800cd9b2e2e7536b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:41:06.066314 9489 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce4f9a3467964a68800cd9b2e2e7536b [term 1 LEADER]: Becoming Leader. State: Replica: ce4f9a3467964a68800cd9b2e2e7536b, State: Running, Role: LEADER
I20260811 18:41:06.066814 9489 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce4f9a3467964a68800cd9b2e2e7536b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4f9a3467964a68800cd9b2e2e7536b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35179 } }
I20260811 18:41:06.067600 9482 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce4f9a3467964a68800cd9b2e2e7536b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:41:06.070464 9493 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce4f9a3467964a68800cd9b2e2e7536b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce4f9a3467964a68800cd9b2e2e7536b. Latest consensus state: current_term: 1 leader_uuid: "ce4f9a3467964a68800cd9b2e2e7536b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4f9a3467964a68800cd9b2e2e7536b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35179 } } }
I20260811 18:41:06.070854 9493 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce4f9a3467964a68800cd9b2e2e7536b [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:06.071990 9490 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce4f9a3467964a68800cd9b2e2e7536b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce4f9a3467964a68800cd9b2e2e7536b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4f9a3467964a68800cd9b2e2e7536b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35179 } } }
I20260811 18:41:06.072315 9490 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce4f9a3467964a68800cd9b2e2e7536b [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:06.072522 9508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:41:06.079483 9508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:41:06.092371 9508 catalog_manager.cc:1347] Generated new cluster ID: 1efceefb853d4fa793a76746354dccf2
I20260811 18:41:06.092514 9508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:41:06.117713 9508 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:41:06.119486 9508 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:41:06.126626 9508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce4f9a3467964a68800cd9b2e2e7536b: Generated new TSK 0
I20260811 18:41:06.127785 9508 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.185.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
I20260811 18:41:06.192649 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:35179
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:37095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:41:06.537645 9532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:06.538002 9532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:06.538180 9532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:06.545825 9532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:06.545995 9532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:06.546176 9532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:41:06.556280 9532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:37095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:35179
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:06.557749 9532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:06.559078 9532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:06.573130 9546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:06.573261 9545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:06.573086 9550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:06.574424 9532 server_base.cc:1034] running on GCE node
I20260811 18:41:06.575089 9532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:06.576036 9532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:06.577283 9532 hybrid_clock.cc:648] HybridClock initialized: now 1786473666577226 us; error 45 us; skew 500 ppm
I20260811 18:41:06.577613 9532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:06 dist-test-slave-z8x9 krb5kdc[9322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473666, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:06.580941 9532 init.cc:377] Logged in from keytab as kudu/127.30.185.129@KRBTEST.COM (short username kudu)
I20260811 18:41:06.582592 9532 webserver.cc:460] Webserver started at http://127.30.185.129:43587/ using document root <none> and password file <none>
I20260811 18:41:06.583298 9532 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:06.583405 9532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:06.583727 9532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:06.586795 9532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "ac5832e333994ec5b766cb240c8280f4"
format_stamp: "Formatted at 2026-08-11 18:41:06 on dist-test-slave-z8x9"
server_key: "2019ac847dd6cc81181fe28ac4c1c218"
server_key_iv: "4816e9c975baee85aa250f35d468f234"
server_key_version: "encryptionkey@0"
I20260811 18:41:06.587548 9532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "ac5832e333994ec5b766cb240c8280f4"
format_stamp: "Formatted at 2026-08-11 18:41:06 on dist-test-slave-z8x9"
server_key: "2019ac847dd6cc81181fe28ac4c1c218"
server_key_iv: "4816e9c975baee85aa250f35d468f234"
server_key_version: "encryptionkey@0"
I20260811 18:41:06.592993 9532 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 18:41:06.596493 9578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:06.597939 9532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:41:06.598124 9532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "ac5832e333994ec5b766cb240c8280f4"
format_stamp: "Formatted at 2026-08-11 18:41:06 on dist-test-slave-z8x9"
server_key: "2019ac847dd6cc81181fe28ac4c1c218"
server_key_iv: "4816e9c975baee85aa250f35d468f234"
server_key_version: "encryptionkey@0"
I20260811 18:41:06.598354 9532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:06.610773 9532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:06.615286 9532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:06.615602 9532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:06.616586 9532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:06.618010 9532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:06.618115 9532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:06.618268 9532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:06.618341 9532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:06.653324 9532 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:34495
I20260811 18:41:06.653419 9794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:34495 every 8 connection(s)
I20260811 18:41:06.654703 9532 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:41:06.662638 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 9532
I20260811 18:41:06.662915 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:41:06.663833 31462 external_mini_cluster.cc:1428] Setting key 0a3386ae57fce6ab3235c8a0eeebe832
Aug 11 18:41:06 dist-test-slave-z8x9 krb5kdc[9322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473666, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:41:06.679427 9799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:06.657681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:36741 (local address 127.30.185.190:35179)
0811 18:41:06.658155 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:06.658160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:06.659146 (+ 986us) server_negotiation.cc:407] Connection header received
0811 18:41:06.662181 (+ 3035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:06.662187 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:06.662357 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:06.662492 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:06.664631 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:06.665819 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:06.667808 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:06.668607 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:06.668775 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:06.672931 (+ 4156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:06.672936 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:06.672940 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:06.672975 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:06.675867 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:06.676886 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:06.676890 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:06.676892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:06.677021 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:06.677596 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:06.677603 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:06.677609 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:06.678033 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:06.678199 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:06.678949 (+ 750us) server_negotiation.cc:299] Negotiation successful
0811 18:41:06.679159 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":241,"threads_started":1}
I20260811 18:41:06.680446 9808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:06.658234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:35179 (local address 127.30.185.129:36741)
0811 18:41:06.658966 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:06.659009 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:06.661774 (+ 2765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:06.662889 (+ 1115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:06.662901 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:06.663780 (+ 879us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:06.664388 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:06.664406 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:06.666047 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:06.666051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:06.667646 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:06.667664 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:06.668867 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:06.668873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:06.669236 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:06.669990 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:06.670014 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:06.672629 (+ 2615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:06.676029 (+ 3400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:06.676042 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:06.676065 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:06.676759 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:06.677142 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:06.677149 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:06.677155 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:06.677460 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:06.678164 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:06.678170 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:06.678532 (+ 362us) client_negotiation.cc:769] Sending connection context
0811 18:41:06.678771 (+ 239us) client_negotiation.cc:240] Negotiation successful
0811 18:41:06.679222 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":295,"threads_started":1}
I20260811 18:41:06.682189 9796 heartbeater.cc:347] Connected to a master server at 127.30.185.190:35179
I20260811 18:41:06.682560 9796 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:06.683456 9796 heartbeater.cc:511] Master 127.30.185.190:35179 requested a full tablet report, sending...
I20260811 18:41:06.685940 9416 ts_manager.cc:194] Registered new tserver with Master: ac5832e333994ec5b766cb240c8280f4 (127.30.185.129:34495)
I20260811 18:41:06.687436 9416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:36741
WARNING: no policy specified for kudu/127.30.185.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
I20260811 18:41:06.728410 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:35179
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:37095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:41:07.073546 9812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:07.073885 9812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:07.074082 9812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:07.081647 9812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:07.081775 9812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:07.081948 9812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:41:07.092473 9812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:37095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:35179
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:07.093997 9812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:07.095387 9812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:07.109598 9822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:07.109686 9819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:07.109685 9825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:07.109889 9812 server_base.cc:1034] running on GCE node
I20260811 18:41:07.110850 9812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:07.111778 9812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:07.113011 9812 hybrid_clock.cc:648] HybridClock initialized: now 1786473667112954 us; error 34 us; skew 500 ppm
I20260811 18:41:07.113396 9812 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:07 dist-test-slave-z8x9 krb5kdc[9322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473667, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:07.116652 9812 init.cc:377] Logged in from keytab as kudu/127.30.185.130@KRBTEST.COM (short username kudu)
I20260811 18:41:07.118209 9812 webserver.cc:460] Webserver started at http://127.30.185.130:41039/ using document root <none> and password file <none>
I20260811 18:41:07.118887 9812 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:07.118991 9812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:07.119269 9812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:07.122366 9812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "63ff85143bc542ba9b8ea073a6d3c08b"
format_stamp: "Formatted at 2026-08-11 18:41:07 on dist-test-slave-z8x9"
server_key: "00a71ecd6352d6a540baecabe42ea7ef"
server_key_iv: "a5fd58346f9ad66c7d77d270b105bffc"
server_key_version: "encryptionkey@0"
I20260811 18:41:07.123111 9812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "63ff85143bc542ba9b8ea073a6d3c08b"
format_stamp: "Formatted at 2026-08-11 18:41:07 on dist-test-slave-z8x9"
server_key: "00a71ecd6352d6a540baecabe42ea7ef"
server_key_iv: "a5fd58346f9ad66c7d77d270b105bffc"
server_key_version: "encryptionkey@0"
I20260811 18:41:07.128499 9812 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 18:41:07.132329 9847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:07.133878 9812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 18:41:07.134065 9812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "63ff85143bc542ba9b8ea073a6d3c08b"
format_stamp: "Formatted at 2026-08-11 18:41:07 on dist-test-slave-z8x9"
server_key: "00a71ecd6352d6a540baecabe42ea7ef"
server_key_iv: "a5fd58346f9ad66c7d77d270b105bffc"
server_key_version: "encryptionkey@0"
I20260811 18:41:07.134316 9812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:07.156973 9812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:07.161557 9812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:07.161886 9812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:07.162950 9812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:07.164420 9812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:07.164552 9812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:07.164711 9812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:07.164767 9812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:07.200618 9812 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:45697
I20260811 18:41:07.200732 10071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:45697 every 8 connection(s)
I20260811 18:41:07.202066 9812 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:41:07.205582 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 9812
I20260811 18:41:07.205818 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:41:07.206418 31462 external_mini_cluster.cc:1428] Setting key 2a8d34e74978fc8f6a90c681ce048dc5
Aug 11 18:41:07 dist-test-slave-z8x9 krb5kdc[9322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473667, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:41:07.228077 10081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:07.204794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:41665 (local address 127.30.185.190:35179)
0811 18:41:07.205371 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:07.205378 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:07.206623 (+ 1245us) server_negotiation.cc:407] Connection header received
0811 18:41:07.208927 (+ 2304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:07.208932 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:07.209035 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:07.209127 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:07.211563 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:07.212482 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:07.215327 (+ 2845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:07.216403 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:07.216549 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:07.221599 (+ 5050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:07.221610 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:07.221619 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:07.221673 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:07.224563 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:07.225535 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:07.225540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:07.225542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:07.225689 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:07.226182 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:07.226187 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:07.226189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:07.226488 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:07.226659 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:07.227414 (+ 755us) server_negotiation.cc:299] Negotiation successful
0811 18:41:07.227668 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":256,"threads_started":1}
I20260811 18:41:07.229262 10082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:07.205368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:35179 (local address 127.30.185.130:41665)
0811 18:41:07.206326 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:07.206407 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:07.208609 (+ 2202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:07.209498 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:07.209511 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:07.210383 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:07.211298 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:07.211337 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:07.212699 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:07.212707 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:07.215053 (+ 2346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:07.215077 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:07.216705 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:07.216716 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:07.217100 (+ 384us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:07.218059 (+ 959us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:07.218097 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:07.221160 (+ 3063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:07.224746 (+ 3586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:07.224758 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:07.224778 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:07.225375 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:07.225818 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:07.225823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:07.225827 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:07.226043 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:07.226631 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:07.226640 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:07.227030 (+ 390us) client_negotiation.cc:769] Sending connection context
0811 18:41:07.227313 (+ 283us) client_negotiation.cc:240] Negotiation successful
0811 18:41:07.227929 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":363,"threads_started":1}
I20260811 18:41:07.231091 10074 heartbeater.cc:347] Connected to a master server at 127.30.185.190:35179
I20260811 18:41:07.231437 10074 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:07.232281 10074 heartbeater.cc:511] Master 127.30.185.190:35179 requested a full tablet report, sending...
I20260811 18:41:07.234087 9416 ts_manager.cc:194] Registered new tserver with Master: 63ff85143bc542ba9b8ea073a6d3c08b (127.30.185.130:45697)
I20260811 18:41:07.235394 9416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:41665
WARNING: no policy specified for kudu/127.30.185.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
I20260811 18:41:07.272814 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:35179
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:37095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:41:07.626428 10093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:07.626788 10093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:07.626968 10093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:07.634315 10093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:07.634487 10093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:07.634685 10093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:41:07.644886 10093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:37095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:35179
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:07.646348 10093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:07.647763 10093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:07.662473 10110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:07.662441 10120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:07.662566 10111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:07.663055 10093 server_base.cc:1034] running on GCE node
I20260811 18:41:07.663724 10093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:07.664686 10093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:07.665933 10093 hybrid_clock.cc:648] HybridClock initialized: now 1786473667665882 us; error 47 us; skew 500 ppm
I20260811 18:41:07.666271 10093 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:07 dist-test-slave-z8x9 krb5kdc[9322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473667, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:07.669785 10093 init.cc:377] Logged in from keytab as kudu/127.30.185.131@KRBTEST.COM (short username kudu)
I20260811 18:41:07.671350 10093 webserver.cc:460] Webserver started at http://127.30.185.131:41285/ using document root <none> and password file <none>
I20260811 18:41:07.672021 10093 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:07.672124 10093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:07.672394 10093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:07.675510 10093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "251db38936c34144b04d7d9081397882"
format_stamp: "Formatted at 2026-08-11 18:41:07 on dist-test-slave-z8x9"
server_key: "1c071d535512763accc23ba576ca9155"
server_key_iv: "19c204cb28a53afd2aeee26d20745d41"
server_key_version: "encryptionkey@0"
I20260811 18:41:07.676295 10093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "251db38936c34144b04d7d9081397882"
format_stamp: "Formatted at 2026-08-11 18:41:07 on dist-test-slave-z8x9"
server_key: "1c071d535512763accc23ba576ca9155"
server_key_iv: "19c204cb28a53afd2aeee26d20745d41"
server_key_version: "encryptionkey@0"
I20260811 18:41:07.681744 10093 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 18:41:07.685596 10131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:07.687031 10093 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:41:07.687215 10093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "251db38936c34144b04d7d9081397882"
format_stamp: "Formatted at 2026-08-11 18:41:07 on dist-test-slave-z8x9"
server_key: "1c071d535512763accc23ba576ca9155"
server_key_iv: "19c204cb28a53afd2aeee26d20745d41"
server_key_version: "encryptionkey@0"
I20260811 18:41:07.687451 10093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:07.691356 9796 heartbeater.cc:503] Master 127.30.185.190:35179 was elected leader, sending a full tablet report...
I20260811 18:41:07.705550 10093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:07.710144 10093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:07.710419 10093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:07.711476 10093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:07.712822 10093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:07.712898 10093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:07.712986 10093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:07.713034 10093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:07.747432 10093 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:33387
I20260811 18:41:07.747483 10357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:33387 every 8 connection(s)
I20260811 18:41:07.748817 10093 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:41:07.751724 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 10093
I20260811 18:41:07.751959 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:41:07.752451 31462 external_mini_cluster.cc:1428] Setting key 362d37797f385c10e6e8118f5ce0bb7f
Aug 11 18:41:07 dist-test-slave-z8x9 krb5kdc[9322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473667, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:41:07.770170 10361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:07.751633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:38487 (local address 127.30.185.190:35179)
0811 18:41:07.752173 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:07.752181 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:07.753553 (+ 1372us) server_negotiation.cc:407] Connection header received
0811 18:41:07.755178 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:07.755183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:07.755306 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:07.755439 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:07.757233 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:07.758045 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:07.759605 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:07.760016 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:07.760114 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:07.763643 (+ 3529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:07.763648 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:07.763651 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:07.763685 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:07.766681 (+ 2996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:07.767582 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:07.767587 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:07.767590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:07.767749 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:07.768349 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:07.768355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:07.768359 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:07.768681 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:07.768847 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:07.769697 (+ 850us) server_negotiation.cc:299] Negotiation successful
0811 18:41:07.769895 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":252,"threads_started":1}
I20260811 18:41:07.771157 10362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:07.752178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:35179 (local address 127.30.185.131:38487)
0811 18:41:07.753095 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:07.753155 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:07.754904 (+ 1749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:07.755710 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:07.755724 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:07.756266 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:07.756993 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:07.757016 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:07.758205 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:07.758209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:07.759468 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:07.759481 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:07.760207 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:07.760211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:07.760418 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:07.761099 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:07.761121 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:07.763433 (+ 2312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:07.766861 (+ 3428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:07.766871 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:07.766889 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:07.767399 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:07.767886 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:07.767894 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:07.767898 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:07.768212 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:07.768881 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:07.768892 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:07.769338 (+ 446us) client_negotiation.cc:769] Sending connection context
0811 18:41:07.769596 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 18:41:07.770022 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":382,"threads_started":1}
I20260811 18:41:07.772823 10359 heartbeater.cc:347] Connected to a master server at 127.30.185.190:35179
I20260811 18:41:07.773307 10359 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:07.774155 10359 heartbeater.cc:511] Master 127.30.185.190:35179 requested a full tablet report, sending...
I20260811 18:41:07.775743 9416 ts_manager.cc:194] Registered new tserver with Master: 251db38936c34144b04d7d9081397882 (127.30.185.131:33387)
I20260811 18:41:07.776598 9416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:38487
I20260811 18:41:07.780957 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/test-admin.keytab.
Aug 11 18:41:07 dist-test-slave-z8x9 krb5kdc[9322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:07.818742 31462 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 18:41:07.824079 10372 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:07.821516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45954 (local address 127.30.185.129:34495)
0811 18:41:07.822163 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:07.822171 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:07.822201 (+ 30us) server_negotiation.cc:407] Connection header received
0811 18:41:07.822340 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:07.822348 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:07.822562 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:07.822699 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:07.823598 (+ 899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45954: BlockingRecv error: recv got EOF from 127.0.0.1:45954 (error 108)
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":329,"threads_started":1}
W20260811 18:41:07.824249 31462 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/krb5cc' of type FILE
Aug 11 18:41:07 dist-test-slave-z8x9 krb5kdc[9322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:07.825901 31462 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 18:41:07 dist-test-slave-z8x9 krb5kdc[9322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:07.827319 31462 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 18:41:07 dist-test-slave-z8x9 krb5kdc[9322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.129@KRBTEST.COM
I20260811 18:41:07.844051 10372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:07.829000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45960 (local address 127.30.185.129:34495)
0811 18:41:07.829337 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:07.829344 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:07.829523 (+ 179us) server_negotiation.cc:407] Connection header received
0811 18:41:07.829670 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:07.829676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:07.829750 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:07.829851 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:07.830794 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:07.831884 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:07.833124 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:07.833837 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:07.833935 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:07.837923 (+ 3988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:07.837938 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:07.837955 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:07.837999 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:07.841238 (+ 3239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:07.841980 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:07.841989 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:07.841991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:07.842181 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:07.842555 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:07.842559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:07.842562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:07.842951 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:07.843217 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:07.843453 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 18:41:07.843755 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260811 18:41:07.850531 31462 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/krb5cc' of type FILE
W20260811 18:41:07.850554 10372 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:07.848510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45962 (local address 127.30.185.129:34495)
0811 18:41:07.848729 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:07.848735 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:07.848904 (+ 169us) server_negotiation.cc:407] Connection header received
0811 18:41:07.849072 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:07.849077 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:07.849257 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:07.849344 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:07.850252 (+ 908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45962: BlockingRecv error: recv got EOF from 127.0.0.1:45962 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 18:41:07 dist-test-slave-z8x9 krb5kdc[9322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:07.852229 31462 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 18:41:07 dist-test-slave-z8x9 krb5kdc[9322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:07.853600 31462 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 18:41:07 dist-test-slave-z8x9 krb5kdc[9322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.129@KRBTEST.COM
I20260811 18:41:07.868510 10372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:07.855040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45964 (local address 127.30.185.129:34495)
0811 18:41:07.855245 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:07.855252 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:07.855583 (+ 331us) server_negotiation.cc:407] Connection header received
0811 18:41:07.855775 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:07.855785 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:07.855853 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:07.855957 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:07.856942 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:07.857771 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:07.858982 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:07.859434 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:07.859570 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:07.863268 (+ 3698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:07.863273 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:07.863277 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:07.863310 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:07.865858 (+ 2548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:07.866503 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:07.866510 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:07.866513 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:07.866642 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:07.867043 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:07.867049 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:07.867053 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:07.867378 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:07.867639 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:07.867956 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 18:41:07.868163 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 18:41:07.872953 10372 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:07.871290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45974 (local address 127.30.185.129:34495)
0811 18:41:07.871504 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:07.871509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:07.871637 (+ 128us) server_negotiation.cc:407] Connection header received
0811 18:41:07.871823 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:07.871828 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:07.871906 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:07.872010 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:07.872709 (+ 699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45974: BlockingRecv error: recv got EOF from 127.0.0.1:45974 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260811 18:41:07.873347 31462 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786473610573616-31462-0/krb5kdc/krb5cc' of type FILE
Aug 11 18:41:07 dist-test-slave-z8x9 krb5kdc[9322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:07.875136 31462 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 18:41:07 dist-test-slave-z8x9 krb5kdc[9322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:07.876483 31462 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 18:41:07 dist-test-slave-z8x9 krb5kdc[9322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.129@KRBTEST.COM
I20260811 18:41:07.891799 10372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:07.877944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45986 (local address 127.30.185.129:34495)
0811 18:41:07.878177 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:07.878182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:07.878473 (+ 291us) server_negotiation.cc:407] Connection header received
0811 18:41:07.878662 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:07.878666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:07.878723 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:07.878826 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:07.879792 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:07.880524 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:07.881853 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:07.882323 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:07.882469 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:07.886409 (+ 3940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:07.886414 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:07.886417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:07.886452 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:07.888893 (+ 2441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:07.889708 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:07.889715 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:07.889718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:07.889866 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:07.890261 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:07.890266 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:07.890268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:07.890566 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:07.890892 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:07.891233 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 18:41:07.891479 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 18:41:07.893476 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 9532
I20260811 18:41:07.906119 10372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:07.894287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46002 (local address 127.30.185.129:34495)
0811 18:41:07.894449 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:07.894453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:07.894923 (+ 470us) server_negotiation.cc:407] Connection header received
0811 18:41:07.895052 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:07.895055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:07.895111 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:07.895187 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:07.896139 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:07.896799 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:07.898167 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:07.898574 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:07.898667 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:07.900775 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:07.900780 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:07.900783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:07.900811 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:07.903459 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:07.904076 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:07.904079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:07.904081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:07.904200 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:07.904587 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:07.904592 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:07.904595 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:07.904906 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:07.905116 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:07.905676 (+ 560us) server_negotiation.cc:299] Negotiation successful
0811 18:41:07.905856 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:41:07.906821 9784 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:07.992395 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 9532
I20260811 18:41:08.019016 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 9812
Aug 11 18:41:08 dist-test-slave-z8x9 krb5kdc[9322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.130@KRBTEST.COM
I20260811 18:41:08.036567 10405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:08.020098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50150 (local address 127.30.185.130:45697)
0811 18:41:08.020761 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:08.020768 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:08.020795 (+ 27us) server_negotiation.cc:407] Connection header received
0811 18:41:08.020856 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:08.020870 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:08.021076 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:08.021272 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:08.022419 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:08.023453 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:08.025403 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:08.025981 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:08.026088 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:08.030773 (+ 4685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:08.030787 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:08.030803 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:08.030848 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:08.033876 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:08.034643 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:08.034651 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:08.034653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:08.034789 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:08.035160 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:08.035164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:08.035166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:08.035531 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:08.035771 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:08.036055 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 18:41:08.036265 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":366,"threads_started":1}
I20260811 18:41:08.038012 10062 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:08.125003 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 9812
I20260811 18:41:08.150803 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 10093
Aug 11 18:41:08 dist-test-slave-z8x9 krb5kdc[9322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.131@KRBTEST.COM
I20260811 18:41:08.168511 10409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:08.151842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53710 (local address 127.30.185.131:33387)
0811 18:41:08.152589 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:08.152596 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:08.152626 (+ 30us) server_negotiation.cc:407] Connection header received
0811 18:41:08.152688 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:08.152694 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:08.152911 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:08.153051 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:08.154290 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:08.155471 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:08.156841 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:08.157689 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:08.157797 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:08.162093 (+ 4296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:08.162109 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:08.162125 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:08.162169 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:08.165426 (+ 3257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:08.166383 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:08.166392 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:08.166394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:08.166537 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:08.166973 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:08.166976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:08.166978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:08.167364 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:08.167617 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:08.167879 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 18:41:08.168209 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":421,"threads_started":1}
I20260811 18:41:08.170053 10348 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:08.258069 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 10093
I20260811 18:41:08.284075 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 9347
I20260811 18:41:08.285411 9471 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:08.389844 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 9347
2026-08-11T18:41:08Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4584 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:41:08 dist-test-slave-z8x9 krb5kdc[10417](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:41:08 dist-test-slave-z8x9 krb5kdc[10417](info): set up 2 sockets
Aug 11 18:41:08 dist-test-slave-z8x9 krb5kdc[10417](info): commencing operation
krb5kdc: starting...
W20260811 18:41:09.837587 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.396s user 0.002s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:41:10 dist-test-slave-z8x9 krb5kdc[10417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:41:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:41:10Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.190.keytab.
Entry for principal oryx/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.190.keytab.
I20260811 18:41:10.121057 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:39547
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.190.keytab
--principal=oryx/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:39547
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:41:10.467291 10444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:10.467700 10444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:10.467881 10444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:10.475312 10444 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:41:10.475487 10444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:10.475603 10444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:10.475718 10444 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:41:10.475811 10444 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:41:10.486574 10444 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:39547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.190.keytab
--principal=oryx/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:39547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:10.488061 10444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:10.489522 10444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:10.502133 10455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:10.502110 10458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:10.502079 10467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:10.502986 10444 server_base.cc:1034] running on GCE node
I20260811 18:41:10.503901 10444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:10.505544 10444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:10.506805 10444 hybrid_clock.cc:648] HybridClock initialized: now 1786473670506758 us; error 39 us; skew 500 ppm
I20260811 18:41:10.507170 10444 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:10 dist-test-slave-z8x9 krb5kdc[10417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473670, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:10.510469 10444 init.cc:377] Logged in from keytab as oryx/127.30.185.190@KRBTEST.COM (short username oryx)
I20260811 18:41:10.512101 10444 webserver.cc:460] Webserver started at http://127.30.185.190:38083/ using document root <none> and password file <none>
I20260811 18:41:10.512845 10444 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:10.513002 10444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:10.513387 10444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:10.516470 10444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "0691bb2c526b42a697988a418b6d8750"
format_stamp: "Formatted at 2026-08-11 18:41:10 on dist-test-slave-z8x9"
server_key: "336f92e4985977fb07282b9643e426c0"
server_key_iv: "9e8725e7c0f5f4863c110e19e50bbee4"
server_key_version: "encryptionkey@0"
I20260811 18:41:10.517387 10444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "0691bb2c526b42a697988a418b6d8750"
format_stamp: "Formatted at 2026-08-11 18:41:10 on dist-test-slave-z8x9"
server_key: "336f92e4985977fb07282b9643e426c0"
server_key_iv: "9e8725e7c0f5f4863c110e19e50bbee4"
server_key_version: "encryptionkey@0"
I20260811 18:41:10.522677 10444 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 18:41:10.526242 10482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:10.527577 10444 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:41:10.527791 10444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "0691bb2c526b42a697988a418b6d8750"
format_stamp: "Formatted at 2026-08-11 18:41:10 on dist-test-slave-z8x9"
server_key: "336f92e4985977fb07282b9643e426c0"
server_key_iv: "9e8725e7c0f5f4863c110e19e50bbee4"
server_key_version: "encryptionkey@0"
I20260811 18:41:10.527995 10444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:10.564065 10444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:10.568414 10444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:10.568764 10444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:10.588275 10444 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:39547
I20260811 18:41:10.588310 10598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:39547 every 8 connection(s)
I20260811 18:41:10.589871 10444 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:41:10.590240 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 10444
I20260811 18:41:10.590516 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:41:10.591426 31462 external_mini_cluster.cc:1428] Setting key 1945b8ceb2735dd12d0201bc69ce0cea
I20260811 18:41:10.594636 10599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 18:41:10 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM
I20260811 18:41:10.608227 10599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750: Bootstrap starting.
I20260811 18:41:10.612190 10599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:10.613521 10599 log.cc:824] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750: Log is configured to *not* fsync() on all Append() calls
I20260811 18:41:10.616093 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:10.593733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48580 (local address 127.30.185.190:39547)
0811 18:41:10.594617 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:10.594633 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:10.594678 (+ 45us) server_negotiation.cc:407] Connection header received
0811 18:41:10.596398 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:10.596419 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:10.596780 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:10.597104 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:10.599045 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:10.600465 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:10.602261 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:10.602913 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:10.603047 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:10.607808 (+ 4761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:10.607819 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:10.607839 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:10.607886 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:10.611562 (+ 3676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:10.612458 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:10.612465 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:10.612471 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:10.612671 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:10.613077 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:10.613080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:10.613082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:10.613782 (+ 700us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:10.614082 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:10.614654 (+ 572us) server_negotiation.cc:299] Negotiation successful
0811 18:41:10.615117 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":445,"threads_started":1}
I20260811 18:41:10.616675 10599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750: No bootstrap required, opened a new log
I20260811 18:41:10.624627 10599 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } }
I20260811 18:41:10.625141 10599 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:10.625314 10599 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0691bb2c526b42a697988a418b6d8750, State: Initialized, Role: FOLLOWER
I20260811 18:41:10.625862 10599 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } }
I20260811 18:41:10.626077 10599 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:41:10.626233 10599 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:41:10.626430 10599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:10.628566 10599 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } }
I20260811 18:41:10.629087 10599 leader_election.cc:302] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0691bb2c526b42a697988a418b6d8750; no voters:
I20260811 18:41:10.629760 10599 leader_election.cc:288] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:41:10.629926 10607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:41:10.630625 10607 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [term 1 LEADER]: Becoming Leader. State: Replica: 0691bb2c526b42a697988a418b6d8750, State: Running, Role: LEADER
I20260811 18:41:10.631212 10607 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } }
I20260811 18:41:10.632021 10599 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:41:10.634078 10609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0691bb2c526b42a697988a418b6d8750. Latest consensus state: current_term: 1 leader_uuid: "0691bb2c526b42a697988a418b6d8750" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } }
I20260811 18:41:10.634402 10609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:10.635314 10614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:41:10.635150 10608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0691bb2c526b42a697988a418b6d8750" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } }
I20260811 18:41:10.635465 10608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:10.641065 10614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:41:10.650122 10614 catalog_manager.cc:1347] Generated new cluster ID: 6cdce79fc4a64402b260ed180c0557fb
I20260811 18:41:10.650249 10614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:41:10.667927 10614 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:41:10.669910 10614 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:41:10.677584 10614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750: Generated new TSK 0
I20260811 18:41:10.678637 10614 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.185.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.185.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.185.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.129.keytab.
Entry for principal oryx/127.30.185.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.129.keytab.
I20260811 18:41:10.748917 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:39547
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.129.keytab
--principal=oryx/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:41:11.096095 10644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:11.096457 10644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:11.096634 10644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:11.104182 10644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:11.104350 10644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:11.104547 10644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:41:11.114737 10644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.129.keytab
--principal=oryx/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:39547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:11.116255 10644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:11.117673 10644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:11.130868 10653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:11.130897 10659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:11.131512 10654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:11.132440 10644 server_base.cc:1034] running on GCE node
I20260811 18:41:11.133221 10644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:11.134200 10644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:11.135448 10644 hybrid_clock.cc:648] HybridClock initialized: now 1786473671135396 us; error 44 us; skew 500 ppm
I20260811 18:41:11.135787 10644 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:11 dist-test-slave-z8x9 krb5kdc[10417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473671, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:11.144134 10644 init.cc:377] Logged in from keytab as oryx/127.30.185.129@KRBTEST.COM (short username oryx)
I20260811 18:41:11.146041 10644 webserver.cc:460] Webserver started at http://127.30.185.129:42791/ using document root <none> and password file <none>
I20260811 18:41:11.146764 10644 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:11.146925 10644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:11.147248 10644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:11.150278 10644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "b92d06495a8d4bd1a82526fe9071c9d1"
format_stamp: "Formatted at 2026-08-11 18:41:11 on dist-test-slave-z8x9"
server_key: "eeff0dc1c4748c3ea6b6b2130f7eb15d"
server_key_iv: "fb078b554ff8b12c5059520a99056ee9"
server_key_version: "encryptionkey@0"
I20260811 18:41:11.151081 10644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "b92d06495a8d4bd1a82526fe9071c9d1"
format_stamp: "Formatted at 2026-08-11 18:41:11 on dist-test-slave-z8x9"
server_key: "eeff0dc1c4748c3ea6b6b2130f7eb15d"
server_key_iv: "fb078b554ff8b12c5059520a99056ee9"
server_key_version: "encryptionkey@0"
I20260811 18:41:11.156451 10644 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:41:11.160156 10672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:11.161612 10644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:41:11.161798 10644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "b92d06495a8d4bd1a82526fe9071c9d1"
format_stamp: "Formatted at 2026-08-11 18:41:11 on dist-test-slave-z8x9"
server_key: "eeff0dc1c4748c3ea6b6b2130f7eb15d"
server_key_iv: "fb078b554ff8b12c5059520a99056ee9"
server_key_version: "encryptionkey@0"
I20260811 18:41:11.162007 10644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:11.201690 10644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:11.206017 10644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:11.206352 10644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:11.207371 10644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:11.208741 10644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:11.208860 10644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:11.209017 10644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:11.209090 10644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:11.244314 10644 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:34579
I20260811 18:41:11.244383 10879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:34579 every 8 connection(s)
I20260811 18:41:11.245704 10644 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:41:11.248102 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 10644
I20260811 18:41:11.248358 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:41:11.249214 31462 external_mini_cluster.cc:1428] Setting key c4d527ebee5ea6148c9c983925549b77
Aug 11 18:41:11 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473671, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM
I20260811 18:41:11.268744 10884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:11.248436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:34529 (local address 127.30.185.190:39547)
0811 18:41:11.248956 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:11.248961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:11.252135 (+ 3174us) server_negotiation.cc:407] Connection header received
0811 18:41:11.252190 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:11.252193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:11.252255 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:11.252336 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:11.254527 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:11.255244 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:11.257746 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:11.258148 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:11.258249 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:11.262615 (+ 4366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:11.262621 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:11.262624 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:11.262655 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:11.265447 (+ 2792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:11.266269 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:11.266275 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:11.266279 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:11.266416 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:11.266882 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:11.266887 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:11.266930 (+ 43us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:11.267261 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:11.267425 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:11.268149 (+ 724us) server_negotiation.cc:299] Negotiation successful
0811 18:41:11.268378 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":268,"threads_started":1}
I20260811 18:41:11.269824 10886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:11.249085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.30.185.129:34529)
0811 18:41:11.249996 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:11.250050 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:11.251849 (+ 1799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:11.252632 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:11.252646 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:11.253509 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:11.254313 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:11.254354 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:11.255410 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:11.255419 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:11.257592 (+ 2173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:11.257611 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:11.258384 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:11.258394 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:11.258720 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:11.259557 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:11.259592 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:11.262341 (+ 2749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:11.265580 (+ 3239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:11.265589 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:11.265605 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:11.266112 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:11.266551 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:11.266558 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:11.266566 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:11.266749 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:11.267414 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:11.267422 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:11.267749 (+ 327us) client_negotiation.cc:769] Sending connection context
0811 18:41:11.268032 (+ 283us) client_negotiation.cc:240] Negotiation successful
0811 18:41:11.268577 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":358,"threads_started":1}
I20260811 18:41:11.271521 10880 heartbeater.cc:347] Connected to a master server at 127.30.185.190:39547
I20260811 18:41:11.271857 10880 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:11.272636 10880 heartbeater.cc:511] Master 127.30.185.190:39547 requested a full tablet report, sending...
I20260811 18:41:11.275117 10517 ts_manager.cc:194] Registered new tserver with Master: b92d06495a8d4bd1a82526fe9071c9d1 (127.30.185.129:34579)
WARNING: no policy specified for oryx/127.30.185.130@KRBTEST.COM; defaulting to no policy
I20260811 18:41:11.277223 10517 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:34529
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.185.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.185.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.130.keytab.
Entry for principal oryx/127.30.185.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.130.keytab.
I20260811 18:41:11.310634 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:39547
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.130.keytab
--principal=oryx/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:41:11.656399 10890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:11.656726 10890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:11.656833 10890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:11.664906 10890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:11.665050 10890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:11.665336 10890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:41:11.676059 10890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.130.keytab
--principal=oryx/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:39547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:11.677806 10890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:11.679239 10890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:11.693039 10898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:11.693076 10902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:11.693113 10900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:11.694258 10890 server_base.cc:1034] running on GCE node
I20260811 18:41:11.694872 10890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:11.695873 10890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:11.697278 10890 hybrid_clock.cc:648] HybridClock initialized: now 1786473671697231 us; error 53 us; skew 500 ppm
I20260811 18:41:11.697608 10890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:11 dist-test-slave-z8x9 krb5kdc[10417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473671, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:11.700968 10890 init.cc:377] Logged in from keytab as oryx/127.30.185.130@KRBTEST.COM (short username oryx)
I20260811 18:41:11.702556 10890 webserver.cc:460] Webserver started at http://127.30.185.130:40637/ using document root <none> and password file <none>
I20260811 18:41:11.703253 10890 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:11.703408 10890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:11.703727 10890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:11.706697 10890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "ce2d62517cd247bd98c6e31ea0f88cc8"
format_stamp: "Formatted at 2026-08-11 18:41:11 on dist-test-slave-z8x9"
server_key: "73bf429f0111a93afa373efd098263c9"
server_key_iv: "3b014e08e6f1fdbf1f7c6026e46f4c7b"
server_key_version: "encryptionkey@0"
I20260811 18:41:11.707494 10890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "ce2d62517cd247bd98c6e31ea0f88cc8"
format_stamp: "Formatted at 2026-08-11 18:41:11 on dist-test-slave-z8x9"
server_key: "73bf429f0111a93afa373efd098263c9"
server_key_iv: "3b014e08e6f1fdbf1f7c6026e46f4c7b"
server_key_version: "encryptionkey@0"
I20260811 18:41:11.712705 10890 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 18:41:11.716233 10916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:11.717641 10890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:41:11.717825 10890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "ce2d62517cd247bd98c6e31ea0f88cc8"
format_stamp: "Formatted at 2026-08-11 18:41:11 on dist-test-slave-z8x9"
server_key: "73bf429f0111a93afa373efd098263c9"
server_key_iv: "3b014e08e6f1fdbf1f7c6026e46f4c7b"
server_key_version: "encryptionkey@0"
I20260811 18:41:11.718040 10890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:11.732471 10890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:11.736769 10890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:11.737074 10890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:11.738185 10890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:11.739526 10890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:11.739631 10890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:11.739784 10890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:11.739864 10890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:11.773957 10890 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:44261
I20260811 18:41:11.774016 11134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:44261 every 8 connection(s)
I20260811 18:41:11.775267 10890 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:41:11.778498 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 10890
I20260811 18:41:11.778729 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:41:11.779322 31462 external_mini_cluster.cc:1428] Setting key 599568b52b3b8310d01d14d723a849e3
Aug 11 18:41:11 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473671, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM
I20260811 18:41:11.796429 11140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:11.777841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:35325 (local address 127.30.185.190:39547)
0811 18:41:11.778359 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:11.778369 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:11.779377 (+ 1008us) server_negotiation.cc:407] Connection header received
0811 18:41:11.781821 (+ 2444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:11.781827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:11.781926 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:11.782045 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:11.783602 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:11.784347 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:11.785997 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:11.786430 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:11.786574 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:11.790161 (+ 3587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:11.790170 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:11.790177 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:11.790224 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:11.793014 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:11.794013 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:11.794017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:11.794019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:11.794149 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:11.794647 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:11.794650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:11.794652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:11.794902 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:11.795011 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:11.795982 (+ 971us) server_negotiation.cc:299] Negotiation successful
0811 18:41:11.796152 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":285,"threads_started":1}
I20260811 18:41:11.798117 11144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:11.778377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.30.185.130:35325)
0811 18:41:11.779207 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:11.779246 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:11.781521 (+ 2275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:11.782301 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:11.782310 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:11.782840 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:11.783422 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:11.783441 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:11.784517 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:11.784524 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:11.785844 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:11.785857 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:11.786669 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:11.786676 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:11.786913 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:11.787636 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:11.787660 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:11.789919 (+ 2259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:11.793197 (+ 3278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:11.793210 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:11.793229 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:11.793883 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:11.794257 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:11.794264 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:11.794270 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:11.794532 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:11.795040 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:11.795052 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:11.795567 (+ 515us) client_negotiation.cc:769] Sending connection context
0811 18:41:11.795889 (+ 322us) client_negotiation.cc:240] Negotiation successful
0811 18:41:11.796487 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":360,"threads_started":1}
I20260811 18:41:11.799659 11135 heartbeater.cc:347] Connected to a master server at 127.30.185.190:39547
I20260811 18:41:11.799978 11135 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:11.800720 11135 heartbeater.cc:511] Master 127.30.185.190:39547 requested a full tablet report, sending...
I20260811 18:41:11.802342 10517 ts_manager.cc:194] Registered new tserver with Master: ce2d62517cd247bd98c6e31ea0f88cc8 (127.30.185.130:44261)
I20260811 18:41:11.803186 10517 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:35325
WARNING: no policy specified for oryx/127.30.185.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.185.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.185.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.131.keytab.
Entry for principal oryx/127.30.185.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.131.keytab.
I20260811 18:41:11.844163 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:39547
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.131.keytab
--principal=oryx/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:41:12.192914 11147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:12.193284 11147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:12.193403 11147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:12.200835 11147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:12.200956 11147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:12.201159 11147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:41:12.211184 11147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.131.keytab
--principal=oryx/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:39547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:12.212532 11147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:12.214116 11147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:12.229691 11156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:12.229771 11152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:12.229816 11154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:12.230496 11147 server_base.cc:1034] running on GCE node
I20260811 18:41:12.231165 11147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:12.232393 11147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:12.233654 11147 hybrid_clock.cc:648] HybridClock initialized: now 1786473672233569 us; error 72 us; skew 500 ppm
I20260811 18:41:12.234127 11147 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:12 dist-test-slave-z8x9 krb5kdc[10417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473672, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:12.238206 11147 init.cc:377] Logged in from keytab as oryx/127.30.185.131@KRBTEST.COM (short username oryx)
I20260811 18:41:12.239794 11147 webserver.cc:460] Webserver started at http://127.30.185.131:38665/ using document root <none> and password file <none>
I20260811 18:41:12.240537 11147 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:12.240676 11147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:12.241014 11147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:12.244164 11147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "659ef4fa0cb14355b6790530f4b73142"
format_stamp: "Formatted at 2026-08-11 18:41:12 on dist-test-slave-z8x9"
server_key: "5bd4b2c4d8e60ea01056fbbfbc9ae32a"
server_key_iv: "83f44d95ed6da9e50b9ee7e6a2d3542f"
server_key_version: "encryptionkey@0"
I20260811 18:41:12.245008 11147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "659ef4fa0cb14355b6790530f4b73142"
format_stamp: "Formatted at 2026-08-11 18:41:12 on dist-test-slave-z8x9"
server_key: "5bd4b2c4d8e60ea01056fbbfbc9ae32a"
server_key_iv: "83f44d95ed6da9e50b9ee7e6a2d3542f"
server_key_version: "encryptionkey@0"
I20260811 18:41:12.250257 11147 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 18:41:12.253978 11172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:12.255376 11147 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:41:12.255569 11147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "659ef4fa0cb14355b6790530f4b73142"
format_stamp: "Formatted at 2026-08-11 18:41:12 on dist-test-slave-z8x9"
server_key: "5bd4b2c4d8e60ea01056fbbfbc9ae32a"
server_key_iv: "83f44d95ed6da9e50b9ee7e6a2d3542f"
server_key_version: "encryptionkey@0"
I20260811 18:41:12.255801 11147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:12.274649 11147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:12.279161 11147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:12.279484 11147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:12.280436 10880 heartbeater.cc:503] Master 127.30.185.190:39547 was elected leader, sending a full tablet report...
I20260811 18:41:12.280575 11147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:12.282025 11147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:12.282140 11147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:12.282327 11147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:12.282407 11147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:12.316057 11147 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:42453
I20260811 18:41:12.316077 11414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:42453 every 8 connection(s)
I20260811 18:41:12.317489 11147 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:41:12.323515 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 11147
I20260811 18:41:12.323747 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:41:12.324369 31462 external_mini_cluster.cc:1428] Setting key 71fe98eef2cc248a3a7cd19596b0c900
Aug 11 18:41:12 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473672, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM
I20260811 18:41:12.341041 11419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:12.320113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:56225 (local address 127.30.185.190:39547)
0811 18:41:12.320591 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:12.320595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:12.321751 (+ 1156us) server_negotiation.cc:407] Connection header received
0811 18:41:12.324151 (+ 2400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:12.324158 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:12.324333 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:12.324683 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:12.326762 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.327945 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.329615 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.330012 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.330108 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:12.334110 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:12.334115 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:12.334119 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:12.334153 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:12.337072 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:12.338133 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:12.338139 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:12.338142 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:12.338310 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:12.338888 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:12.338893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:12.338902 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:12.339324 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:12.339491 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:12.340363 (+ 872us) server_negotiation.cc:299] Negotiation successful
0811 18:41:12.340610 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":226,"threads_started":1}
I20260811 18:41:12.342170 11420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:12.320692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.30.185.131:56225)
0811 18:41:12.321541 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:12.321582 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:12.323756 (+ 2174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:12.324933 (+ 1177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:12.324948 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:12.325688 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:12.326568 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:12.326594 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.328102 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.328106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:12.329488 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:12.329500 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.330231 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.330236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:12.330436 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:12.331136 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:12.331159 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:12.333856 (+ 2697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:12.337328 (+ 3472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:12.337335 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:12.337351 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:12.337971 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:12.338458 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:12.338463 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:12.338466 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:12.338746 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:12.339459 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:12.339466 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:12.339943 (+ 477us) client_negotiation.cc:769] Sending connection context
0811 18:41:12.340236 (+ 293us) client_negotiation.cc:240] Negotiation successful
0811 18:41:12.340895 (+ 659us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":302,"threads_started":1}
I20260811 18:41:12.343755 11416 heartbeater.cc:347] Connected to a master server at 127.30.185.190:39547
I20260811 18:41:12.344097 11416 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:12.344902 11416 heartbeater.cc:511] Master 127.30.185.190:39547 requested a full tablet report, sending...
I20260811 18:41:12.346570 10517 ts_manager.cc:194] Registered new tserver with Master: 659ef4fa0cb14355b6790530f4b73142 (127.30.185.131:42453)
I20260811 18:41:12.347407 10517 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:56225
I20260811 18:41:12.353068 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.185.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.185.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.185.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.189.keytab.
Entry for principal oryx/127.30.185.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.189.keytab.
I20260811 18:41:12.419117 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.189:41891
--webserver_interface=127.30.185.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.189.keytab
--principal=oryx/127.30.185.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:39547,127.30.185.189:41891
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:41:12.769331 11423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:12.769695 11423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:12.769865 11423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:12.777537 11423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:41:12.777676 11423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:12.777798 11423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:12.777912 11423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:41:12.777992 11423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:41:12.788205 11423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:39547,127.30.185.189:41891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.189.keytab
--principal=oryx/127.30.185.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.189:41891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.185.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
W20260811 18:41:12.788643 11423 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.30.185.190:39547,127.30.185.189:41891'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 18:41:12.789821 11423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:12.791162 11423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:12.803660 11431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:12.803675 11430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:12.803911 11437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:12.804463 11423 server_base.cc:1034] running on GCE node
I20260811 18:41:12.805567 11423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:12.805763 11135 heartbeater.cc:503] Master 127.30.185.190:39547 was elected leader, sending a full tablet report...
I20260811 18:41:12.807297 11423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:12.808573 11423 hybrid_clock.cc:648] HybridClock initialized: now 1786473672808496 us; error 46 us; skew 500 ppm
I20260811 18:41:12.808951 11423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:12 dist-test-slave-z8x9 krb5kdc[10417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473672, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:12.812410 11423 init.cc:377] Logged in from keytab as oryx/127.30.185.189@KRBTEST.COM (short username oryx)
I20260811 18:41:12.813983 11423 webserver.cc:460] Webserver started at http://127.30.185.189:40141/ using document root <none> and password file <none>
I20260811 18:41:12.814687 11423 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:12.814779 11423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:12.815060 11423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:12.818195 11423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/data/instance:
uuid: "ef1b11a131674161bb732b9c0300aae9"
format_stamp: "Formatted at 2026-08-11 18:41:12 on dist-test-slave-z8x9"
server_key: "6263ed883cd03acf44f5a0d128fc686a"
server_key_iv: "aca390e6b76c574b5719c4d377e323f2"
server_key_version: "encryptionkey@0"
I20260811 18:41:12.819048 11423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/wal/instance:
uuid: "ef1b11a131674161bb732b9c0300aae9"
format_stamp: "Formatted at 2026-08-11 18:41:12 on dist-test-slave-z8x9"
server_key: "6263ed883cd03acf44f5a0d128fc686a"
server_key_iv: "aca390e6b76c574b5719c4d377e323f2"
server_key_version: "encryptionkey@0"
I20260811 18:41:12.824726 11423 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:41:12.828301 11452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:12.829854 11423 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:41:12.830066 11423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/wal
uuid: "ef1b11a131674161bb732b9c0300aae9"
format_stamp: "Formatted at 2026-08-11 18:41:12 on dist-test-slave-z8x9"
server_key: "6263ed883cd03acf44f5a0d128fc686a"
server_key_iv: "aca390e6b76c574b5719c4d377e323f2"
server_key_version: "encryptionkey@0"
I20260811 18:41:12.830304 11423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:12.845029 11423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:12.849519 11423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:12.849828 11423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:12.869024 11423 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.189:41891
I20260811 18:41:12.869030 11540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.189:41891 every 8 connection(s)
I20260811 18:41:12.870594 11423 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/data/info.pb
I20260811 18:41:12.874475 11542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:12.878002 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 11423
I20260811 18:41:12.882524 11542 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:41:12 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.185.189@KRBTEST.COM
Aug 11 18:41:12 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473672, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.189@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM
I20260811 18:41:12.901507 11549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:12.885800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42344 (local address 127.30.185.190:39547)
0811 18:41:12.886395 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:12.886404 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:12.886859 (+ 455us) server_negotiation.cc:407] Connection header received
0811 18:41:12.887080 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:12.887087 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:12.887160 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:12.887279 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:12.888740 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.889678 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.891257 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.891966 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.892109 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:12.895654 (+ 3545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:12.895659 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:12.895662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:12.895698 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:12.898401 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:12.899091 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:12.899095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:12.899097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:12.899216 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:12.899626 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:12.899631 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:12.899637 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:12.899992 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:12.900108 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:12.900818 (+ 710us) server_negotiation.cc:299] Negotiation successful
0811 18:41:12.901091 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":288,"threads_started":1}
I20260811 18:41:12.902774 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:12.880052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43768 (local address 127.30.185.189:41891)
0811 18:41:12.880926 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:12.880939 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:12.880979 (+ 40us) server_negotiation.cc:407] Connection header received
0811 18:41:12.882758 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:12.882782 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:12.883192 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:12.883555 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:12.885542 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.887462 (+ 1920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.889201 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.889987 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.890185 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:12.894971 (+ 4786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:12.894982 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:12.895004 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:12.895049 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:12.898304 (+ 3255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:12.899020 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:12.899031 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:12.899038 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:12.899197 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:12.899643 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:12.899647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:12.899649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:12.900216 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:12.900516 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:12.901232 (+ 716us) server_negotiation.cc:299] Negotiation successful
0811 18:41:12.901456 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":458,"threads_started":1}
I20260811 18:41:12.902755 11552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:12.885912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.0.0.1:42344)
0811 18:41:12.886712 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:12.886748 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:12.886948 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:12.887343 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:12.887350 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:12.888238 (+ 888us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:12.888563 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:12.888575 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.889831 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.889836 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:12.891103 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:12.891116 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.892198 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.892202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:12.892325 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:12.892990 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:12.893012 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:12.895457 (+ 2445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:12.898529 (+ 3072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:12.898538 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:12.898547 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:12.898936 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:12.899359 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:12.899362 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:12.899364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:12.899507 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:12.900127 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:12.900135 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:12.900356 (+ 221us) client_negotiation.cc:769] Sending connection context
0811 18:41:12.900697 (+ 341us) client_negotiation.cc:240] Negotiation successful
0811 18:41:12.901247 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":421,"threads_started":1}
I20260811 18:41:12.905138 11542 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:41:12 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473672, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.189@KRBTEST.COM for oryx/127.30.185.189@KRBTEST.COM
I20260811 18:41:12.917387 11552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:12.906069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:41891 (local address 127.0.0.1:43772)
0811 18:41:12.906261 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:12.906279 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:12.906376 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:12.906711 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:12.906714 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:12.907043 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:12.907171 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:12.907178 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.908104 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.908107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:12.908959 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:12.908969 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.909696 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.909700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:12.909865 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:12.910479 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:12.910497 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:12.912509 (+ 2012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:12.915246 (+ 2737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:12.915249 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:12.915252 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:12.915609 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:12.915937 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:12.915940 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:12.915942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:12.916066 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:12.916596 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:12.916599 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:12.916712 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 18:41:12.916820 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 18:41:12.917049 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 18:41:12.917543 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:12.906338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43772 (local address 127.30.185.189:41891)
0811 18:41:12.906487 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:12.906491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:12.906506 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:41:12.906545 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:12.906548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:12.906598 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:12.906676 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:12.907288 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.907980 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.909076 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.909541 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.909643 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:12.912685 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:12.912690 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:12.912693 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:12.912722 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:12.915123 (+ 2401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:12.915712 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:12.915716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:12.915718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:12.915836 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:12.916155 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:12.916158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:12.916160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:12.916479 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:12.916594 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:12.916905 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 18:41:12.917243 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":55}
I20260811 18:41:12.922474 11542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9: Bootstrap starting.
I20260811 18:41:12.925587 11542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:12.926638 11542 log.cc:824] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9: Log is configured to *not* fsync() on all Append() calls
I20260811 18:41:12.929314 11542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9: No bootstrap required, opened a new log
I20260811 18:41:12.937431 11542 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } }
I20260811 18:41:12.937839 11542 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:12.937968 11542 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef1b11a131674161bb732b9c0300aae9, State: Initialized, Role: FOLLOWER
I20260811 18:41:12.938613 11542 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } }
I20260811 18:41:12.939507 11555 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } } }
I20260811 18:41:12.939811 11555 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:41:12.940536 11542 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 18:41:12.949978 11576 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:41:12.950139 11576 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:41:12.953641 11423 master_runner.cc:428] Detected that this master ef1b11a131674161bb732b9c0300aae9 is joining an existing cluster
I20260811 18:41:12.953754 11423 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.185.189:41891
I20260811 18:41:12.977270 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:12.959272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43780 (local address 127.30.185.189:41891)
0811 18:41:12.959484 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:12.959490 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:12.960541 (+ 1051us) server_negotiation.cc:407] Connection header received
0811 18:41:12.960751 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:12.960756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:12.960856 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:12.960923 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:12.962602 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.963762 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.966098 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.967021 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.967186 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:12.969694 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:12.969702 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:12.969708 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:12.969748 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:12.974134 (+ 4386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:12.974985 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:12.974990 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:12.974994 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:12.975142 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:12.975561 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:12.975565 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:12.975569 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:12.975938 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:12.976091 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:12.976584 (+ 493us) server_negotiation.cc:299] Negotiation successful
0811 18:41:12.976835 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":63}
I20260811 18:41:12.978363 11587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:12.959730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:41891 (local address 127.0.0.1:43780)
0811 18:41:12.960451 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:12.960470 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:12.960623 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:12.961203 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:12.961207 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:12.961994 (+ 787us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:12.962442 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:12.962456 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.964013 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.964018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:12.965945 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:12.965962 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.967252 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.967259 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:12.967443 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:12.968192 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:12.968223 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:12.969514 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:12.974278 (+ 4764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:12.974283 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:12.974287 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:12.974862 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:12.975275 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:12.975279 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:12.975282 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:12.975451 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:12.976156 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:12.976161 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:12.976325 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 18:41:12.977567 (+ 1242us) client_negotiation.cc:240] Negotiation successful
0811 18:41:12.977901 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":30,"thread_start_us":370,"threads_started":1}
I20260811 18:41:12.981267 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:12.958178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.0.0.1:42358)
0811 18:41:12.958928 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:12.958950 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:12.959079 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:12.961119 (+ 2040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:12.961127 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:12.962080 (+ 953us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:12.962537 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:12.962553 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.964156 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.964173 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:12.967946 (+ 3773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:12.967999 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.969755 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.969762 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:12.970091 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:12.971052 (+ 961us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:12.971087 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:12.972749 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:12.977658 (+ 4909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:12.977666 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:12.977671 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:12.978218 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:12.978848 (+ 630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:12.978855 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:12.978862 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:12.979048 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:12.979837 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:12.979844 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:12.980095 (+ 251us) client_negotiation.cc:769] Sending connection context
0811 18:41:12.980305 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 18:41:12.980708 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"mutex_wait_us":43,"thread_start_us":401,"threads_started":1}
I20260811 18:41:12.982007 11549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:12.958156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42358 (local address 127.30.185.190:39547)
0811 18:41:12.960549 (+ 2393us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:12.960558 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:12.960581 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:41:12.960648 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:12.960656 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:12.960761 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:12.960868 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:12.962739 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.963889 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.968206 (+ 4317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.969129 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.969358 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:12.973052 (+ 3694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:12.973060 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:12.973064 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:12.973122 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:12.977495 (+ 4373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:12.978406 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:12.978542 (+ 136us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:12.978547 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:12.978707 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:12.979182 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:12.979189 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:12.979195 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:12.979620 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:12.979770 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:12.981388 (+ 1618us) server_negotiation.cc:299] Negotiation successful
0811 18:41:12.981653 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2209}
WARNING: no policy specified for oryx/127.30.185.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.185.188@KRBTEST.COM" created.
Aug 11 18:41:12 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473670, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.190@KRBTEST.COM for oryx/127.30.185.189@KRBTEST.COM
I20260811 18:41:13.009967 11592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:12.989732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:41891 (local address 127.0.0.1:43796)
0811 18:41:12.990460 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:12.990518 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:12.990712 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:12.991342 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:12.991353 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:12.992365 (+ 1012us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:12.992695 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:12.992708 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.994186 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.994191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:12.995542 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:12.995555 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.996377 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.996383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:12.996584 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:12.997677 (+ 1093us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:12.997701 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:13.000761 (+ 3060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:13.005934 (+ 5173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:13.005956 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:13.005968 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:13.006683 (+ 715us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:13.007291 (+ 608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:13.007298 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:13.007305 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:13.007533 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:13.008394 (+ 861us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:13.008406 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:13.008664 (+ 258us) client_negotiation.cc:769] Sending connection context
0811 18:41:13.008904 (+ 240us) client_negotiation.cc:240] Negotiation successful
0811 18:41:13.009384 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":421,"threads_started":1}
I20260811 18:41:13.010332 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:12.990125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43796 (local address 127.30.185.189:41891)
0811 18:41:12.990914 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:12.990920 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:12.990940 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:41:12.991029 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:12.991034 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:12.991142 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:12.991258 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:12.992897 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.994023 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.995692 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:12.996200 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:12.996364 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:13.001020 (+ 4656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:13.001029 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:13.001036 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:13.001087 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:13.005699 (+ 4612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.006861 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.006870 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.006876 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.007116 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.007685 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.007692 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.007698 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.008147 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:13.008305 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:13.009386 (+ 1081us) server_negotiation.cc:299] Negotiation successful
0811 18:41:13.009652 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657}
I20260811 18:41:13.012938 10517 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.185.189:41891: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ef1b11a131674161bb732b9c0300aae9"
member_type: NON_VOTER
last_known_addr {
host: "127.30.185.189"
port: 41891
}
attrs {
promote: true
}
}
dest_uuid: "0691bb2c526b42a697988a418b6d8750"
cas_config_opid_index: -1
I20260811 18:41:13.014716 10517 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: NON_VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: true } }
I20260811 18:41:13.026264 11500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:13.027858 11602 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ef1b11a131674161bb732b9c0300aae9 (127.30.185.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: NON_VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: true } } }
I20260811 18:41:13.030637 11500 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 1 FOLLOWER]: Refusing update from remote peer 0691bb2c526b42a697988a418b6d8750: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 18:41:13.032891 11604 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: NON_VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:41:13.034011 11602 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.185.189:41891
I20260811 18:41:13.035255 11423 master.cc:537] Master@127.30.185.189:41891 shutting down...
W20260811 18:41:13.038177 10487 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.185.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.188.keytab.
Entry for principal oryx/127.30.185.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.188.keytab.
I20260811 18:41:13.042279 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.188:37311
--webserver_interface=127.30.185.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.188.keytab
--principal=oryx/127.30.185.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:39547,127.30.185.189:41891,127.30.185.188:37311
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:41:13.046276 11555 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:41:13.046635 11555 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } }
I20260811 18:41:13.048462 11555 leader_election.cc:288] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0691bb2c526b42a697988a418b6d8750 (127.30.185.190:39547)
I20260811 18:41:13.049345 11603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0691bb2c526b42a697988a418b6d8750" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: NON_VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: true } } }
I20260811 18:41:13.049715 11603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:13.054152 10544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ef1b11a131674161bb732b9c0300aae9" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0691bb2c526b42a697988a418b6d8750" is_pre_election: true
W20260811 18:41:13.054888 10487 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 -> Peer ef1b11a131674161bb732b9c0300aae9 (127.30.185.189:41891): Couldn't send request to peer ef1b11a131674161bb732b9c0300aae9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:41:13.055573 11462 leader_election.cc:302] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: ef1b11a131674161bb732b9c0300aae9; no voters: 0691bb2c526b42a697988a418b6d8750
I20260811 18:41:13.056041 11555 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:41:13.070801 11423 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:41:13.071303 11423 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:41:13.071553 11423 tablet_replica.cc:330] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9: stopping tablet replica
I20260811 18:41:13.079977 11423 master.cc:552] Master@127.30.185.189:41891 shutdown complete.
I20260811 18:41:13.080132 11423 master_runner.cc:305] Clearing existing system tablet
I20260811 18:41:13.080621 11423 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:13.080731 11423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:13.083885 11423 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 18:41:13.085727 11615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:13.086081 11423 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:13.086462 11423 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 18:41:13.086587 11423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/wal
uuid: "ef1b11a131674161bb732b9c0300aae9"
format_stamp: "Formatted at 2026-08-11 18:41:12 on dist-test-slave-z8x9"
server_key: "6263ed883cd03acf44f5a0d128fc686a"
server_key_iv: "aca390e6b76c574b5719c4d377e323f2"
server_key_version: "encryptionkey@0"
I20260811 18:41:13.092166 11423 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:41:13.099447 11423 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 18:41:13.099720 11423 log.cc:1196] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 18:41:13.100232 11423 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9: Deleting consensus metadata
I20260811 18:41:13.100593 11423 master_runner.cc:315] Copying system tablet from 127.30.185.190:39547
I20260811 18:41:13.103745 11423 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9: tablet copy: Beginning tablet copy session from remote peer at address 127.30.185.190:39547
I20260811 18:41:13.125587 11549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:13.104896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42370 (local address 127.30.185.190:39547)
0811 18:41:13.105090 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:13.105097 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:13.109847 (+ 4750us) server_negotiation.cc:407] Connection header received
0811 18:41:13.110122 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:13.110130 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:13.110247 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:13.110409 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:13.111513 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.112874 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.114868 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.115564 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.115703 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:13.118212 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:13.118218 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:13.118223 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:13.118266 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:13.122232 (+ 3966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.123192 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.123198 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.123201 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.123395 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.123859 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.123864 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.123867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.124256 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:13.124448 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:13.124918 (+ 470us) server_negotiation.cc:299] Negotiation successful
0811 18:41:13.125158 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 18:41:13.125626 11623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:13.104862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.0.0.1:42370)
0811 18:41:13.109660 (+ 4798us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:13.109684 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:13.109882 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:13.110449 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:13.110455 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:13.111007 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:13.111294 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:13.111306 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.113058 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.113064 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:13.114677 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:13.114693 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.115773 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.115779 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:13.115989 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:13.116743 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:13.116768 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:13.117978 (+ 1210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:13.122411 (+ 4433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:13.122417 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:13.122422 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:13.123033 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:13.123522 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:13.123527 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:13.123531 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:13.123719 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:13.124397 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:13.124403 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:13.124588 (+ 185us) client_negotiation.cc:769] Sending connection context
0811 18:41:13.124751 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 18:41:13.125097 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4630,"thread_start_us":4508,"threads_started":1}
I20260811 18:41:13.126974 10564 tablet_copy_service.cc:140] P 0691bb2c526b42a697988a418b6d8750: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ef1b11a131674161bb732b9c0300aae9 ({username='oryx', principal='oryx/127.30.185.189@KRBTEST.COM'} at 127.0.0.1:42370)
I20260811 18:41:13.127389 10564 tablet_copy_service.cc:161] P 0691bb2c526b42a697988a418b6d8750: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ef1b11a131674161bb732b9c0300aae9 at {username='oryx', principal='oryx/127.30.185.189@KRBTEST.COM'} at 127.0.0.1:42370: session id = ef1b11a131674161bb732b9c0300aae9-00000000000000000000000000000000
I20260811 18:41:13.132289 10564 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 18:41:13.135903 11423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 18:41:13.143391 11423 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9: tablet copy: Starting download of 0 data blocks...
I20260811 18:41:13.143826 11423 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9: tablet copy: Starting download of 1 WAL segments...
I20260811 18:41:13.148079 11423 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 18:41:13.153334 10564 tablet_copy_service.cc:342] P 0691bb2c526b42a697988a418b6d8750: Request end of tablet copy session ef1b11a131674161bb732b9c0300aae9-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.185.189@KRBTEST.COM'} at 127.0.0.1:42370
I20260811 18:41:13.153748 10564 tablet_copy_service.cc:433] P 0691bb2c526b42a697988a418b6d8750: ending tablet copy session ef1b11a131674161bb732b9c0300aae9-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ef1b11a131674161bb732b9c0300aae9
I20260811 18:41:13.166769 11423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:13.169896 11624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:13.170205 11625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:13.177603 11628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:13.178138 11423 server_base.cc:1034] running on GCE node
I20260811 18:41:13.178575 11423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:13.179354 11423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:13.180650 11423 hybrid_clock.cc:648] HybridClock initialized: now 1786473673180569 us; error 49 us; skew 500 ppm
I20260811 18:41:13.180986 11423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:13 dist-test-slave-z8x9 krb5kdc[10417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473673, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:13.183442 11423 init.cc:377] Logged in from keytab as oryx/127.30.185.189@KRBTEST.COM (short username oryx)
I20260811 18:41:13.184729 11423 webserver.cc:460] Webserver started at http://127.30.185.189:38513/ using document root <none> and password file <none>
I20260811 18:41:13.185122 11423 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:13.185249 11423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:13.188583 11423 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.002s
I20260811 18:41:13.190769 11644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:13.191227 11423 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:41:13.191372 11423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/wal
uuid: "ef1b11a131674161bb732b9c0300aae9"
format_stamp: "Formatted at 2026-08-11 18:41:12 on dist-test-slave-z8x9"
server_key: "6263ed883cd03acf44f5a0d128fc686a"
server_key_iv: "aca390e6b76c574b5719c4d377e323f2"
server_key_version: "encryptionkey@0"
I20260811 18:41:13.191586 11423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:13.214520 11423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:13.219430 11423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:13.267385 11423 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.189:41891
I20260811 18:41:13.267412 11730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.189:41891 every 8 connection(s)
I20260811 18:41:13.268491 11423 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-1/data/info.pb
I20260811 18:41:13.275425 11733 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:41:13.279731 11733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9: Bootstrap starting.
I20260811 18:41:13.302924 11733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:41:13.303620 11733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9: Bootstrap complete.
I20260811 18:41:13.304787 11733 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: NON_VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: true } }
I20260811 18:41:13.305085 11733 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ef1b11a131674161bb732b9c0300aae9, State: Initialized, Role: LEARNER
I20260811 18:41:13.305380 11733 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: NON_VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: true } }
I20260811 18:41:13.306282 11741 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: NON_VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: true } } }
I20260811 18:41:13.306624 11733 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:41:13.306769 11741 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [sys.catalog]: This master's current role is: LEARNER
I20260811 18:41:13.322448 11766 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 18:41:13.350126 11416 heartbeater.cc:503] Master 127.30.185.190:39547 was elected leader, sending a full tablet report...
W20260811 18:41:13.500384 11612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:13.500833 11612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:13.501024 11612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:13.510649 11612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:41:13.510847 11612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:13.510983 11612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:13.511101 11612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:41:13.511197 11612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:41:13.523727 11612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:39547,127.30.185.189:41891,127.30.185.188:37311
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.188.keytab
--principal=oryx/127.30.185.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.188:37311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.185.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:13.525414 11612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:13.526890 11612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:13.541706 11773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:13.541746 11784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:13.541817 11781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:13.542773 11612 server_base.cc:1034] running on GCE node
I20260811 18:41:13.543833 11612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:13.545426 11612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:13.546651 11612 hybrid_clock.cc:648] HybridClock initialized: now 1786473673546599 us; error 42 us; skew 500 ppm
I20260811 18:41:13.547032 11612 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:13 dist-test-slave-z8x9 krb5kdc[10417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473673, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:13.551345 11612 init.cc:377] Logged in from keytab as oryx/127.30.185.188@KRBTEST.COM (short username oryx)
I20260811 18:41:13.552968 11612 webserver.cc:460] Webserver started at http://127.30.185.188:43283/ using document root <none> and password file <none>
I20260811 18:41:13.553818 11612 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:13.553970 11612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:13.554329 11612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:13.557560 11612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/data/instance:
uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc"
format_stamp: "Formatted at 2026-08-11 18:41:13 on dist-test-slave-z8x9"
server_key: "88e80bc6055ef657f697331ca3722eb2"
server_key_iv: "72391819504e33c90d3bc357cd07433c"
server_key_version: "encryptionkey@0"
I20260811 18:41:13.558390 11612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/wal/instance:
uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc"
format_stamp: "Formatted at 2026-08-11 18:41:13 on dist-test-slave-z8x9"
server_key: "88e80bc6055ef657f697331ca3722eb2"
server_key_iv: "72391819504e33c90d3bc357cd07433c"
server_key_version: "encryptionkey@0"
I20260811 18:41:13.564543 11612 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 18:41:13.568722 11800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:13.570401 11612 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 18:41:13.570626 11612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/wal
uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc"
format_stamp: "Formatted at 2026-08-11 18:41:13 on dist-test-slave-z8x9"
server_key: "88e80bc6055ef657f697331ca3722eb2"
server_key_iv: "72391819504e33c90d3bc357cd07433c"
server_key_version: "encryptionkey@0"
I20260811 18:41:13.570909 11612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:13.621227 11612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:13.622767 11808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:13.592786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:41891 (local address 127.0.0.1:43810)
0811 18:41:13.609224 (+ 16438us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:13.609290 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:13.609558 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:13.610757 (+ 1199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:13.610762 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:13.611211 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:13.611392 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:13.611401 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.612754 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.612758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:13.613766 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:13.613779 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.614667 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.614675 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:13.614843 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:13.615518 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:13.615537 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:13.616330 (+ 793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:13.619617 (+ 3287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:13.619623 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:13.619627 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:13.620166 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:13.620711 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:13.620717 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:13.620720 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:13.620906 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:13.621627 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:13.621632 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:13.621843 (+ 211us) client_negotiation.cc:769] Sending connection context
0811 18:41:13.622024 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 18:41:13.622339 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16152,"thread_start_us":17299,"threads_started":1}
I20260811 18:41:13.622915 11812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:13.592946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43810 (local address 127.30.185.189:41891)
0811 18:41:13.610241 (+ 17295us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:13.610248 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:13.610274 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:41:13.610485 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:13.610491 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:13.610579 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:13.610713 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:13.611557 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.612563 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.613916 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.614493 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.614634 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:13.616543 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:13.616548 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:13.616552 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:13.616584 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:13.619452 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.620312 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.620318 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.620333 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.620587 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.621032 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.621036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.621057 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.621508 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:13.621679 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:13.622153 (+ 474us) server_negotiation.cc:299] Negotiation successful
0811 18:41:13.622384 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17113,"thread_start_us":16974,"threads_started":1}
I20260811 18:41:13.623988 11689 raft_consensus.cc:1203] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260811 18:41:13.624584 11741 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0691bb2c526b42a697988a418b6d8750. Latest consensus state: current_term: 1 leader_uuid: "0691bb2c526b42a697988a418b6d8750" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: NON_VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: true } } }
I20260811 18:41:13.624883 11741 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [sys.catalog]: This master's current role is: LEARNER
I20260811 18:41:13.626735 11612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:13.627095 11612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:13.650167 11612 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.188:37311
I20260811 18:41:13.650198 11908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.188:37311 every 8 connection(s)
I20260811 18:41:13.651647 11612 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/data/info.pb
I20260811 18:41:13.655761 11910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:13.661111 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 11612
I20260811 18:41:13.663611 11910 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:41:13 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473673, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.188@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM
Aug 11 18:41:13 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.185.188@KRBTEST.COM
I20260811 18:41:13.684161 11923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:13.666565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42376 (local address 127.30.185.190:39547)
0811 18:41:13.667189 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:13.667196 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:13.667775 (+ 579us) server_negotiation.cc:407] Connection header received
0811 18:41:13.668140 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:13.668146 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:13.668227 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:13.668344 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:13.669715 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.670476 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.672070 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.672522 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.672656 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:13.677510 (+ 4854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:13.677518 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:13.677523 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:13.677562 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:13.680777 (+ 3215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.681794 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.681797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.681800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.681933 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.682427 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.682430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.682432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.682678 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:13.682835 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:13.683537 (+ 702us) server_negotiation.cc:299] Negotiation successful
0811 18:41:13.683776 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":357,"threads_started":1}
I20260811 18:41:13.685426 11924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:13.666567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.0.0.1:42376)
0811 18:41:13.667611 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:13.667656 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:13.667915 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:13.668420 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:13.668427 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:13.669086 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:13.669540 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:13.669558 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.670659 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.670666 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:13.671926 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:13.671938 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.672769 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.672781 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:13.673001 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:13.673855 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:13.673893 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:13.677220 (+ 3327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:13.680919 (+ 3699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:13.680933 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:13.680948 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:13.681671 (+ 723us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:13.682047 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:13.682056 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:13.682060 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:13.682308 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:13.682808 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:13.682818 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:13.683150 (+ 332us) client_negotiation.cc:769] Sending connection context
0811 18:41:13.683442 (+ 292us) client_negotiation.cc:240] Negotiation successful
0811 18:41:13.683906 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":842,"thread_start_us":683,"threads_started":1}
I20260811 18:41:13.685847 11921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:13.663588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53206 (local address 127.30.185.188:37311)
0811 18:41:13.664789 (+ 1201us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:13.664801 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:13.664862 (+ 61us) server_negotiation.cc:407] Connection header received
0811 18:41:13.667466 (+ 2604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:13.667502 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:13.667944 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:13.668205 (+ 261us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:13.670072 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.671265 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.672584 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.673112 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.673297 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:13.678989 (+ 5692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:13.678998 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:13.679010 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:13.679040 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:13.682680 (+ 3640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.683383 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.683394 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.683397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.683560 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.684001 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.684006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.684009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.684453 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:13.684744 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:13.685116 (+ 372us) server_negotiation.cc:299] Negotiation successful
0811 18:41:13.685414 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":922,"thread_start_us":580,"threads_started":1}
I20260811 18:41:13.687311 11910 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:41:13 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473673, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.188@KRBTEST.COM for oryx/127.30.185.189@KRBTEST.COM
I20260811 18:41:13.701084 11812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:13.688460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43826 (local address 127.30.185.189:41891)
0811 18:41:13.688770 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:13.688776 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:13.688935 (+ 159us) server_negotiation.cc:407] Connection header received
0811 18:41:13.689109 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:13.689115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:13.689264 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:13.689348 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:13.690041 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.690770 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.692098 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.692529 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.692623 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:13.695814 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:13.695819 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:13.695823 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:13.695856 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:13.698390 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.699036 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.699048 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.699050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.699185 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.699559 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.699564 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.699567 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.699811 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:13.699980 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:13.700385 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 18:41:13.700671 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260811 18:41:13.701229 11924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:13.688487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:41891 (local address 127.0.0.1:43826)
0811 18:41:13.688825 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:13.688845 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:13.688974 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:13.689400 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:13.689404 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:13.689719 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:13.689918 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:13.689927 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.690920 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.690928 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:13.691968 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:13.691981 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.692651 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.692656 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:13.692846 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:13.693439 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:13.693456 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:13.695640 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:13.698527 (+ 2887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:13.698532 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:13.698535 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:13.698913 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:13.699299 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:13.699303 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:13.699307 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:13.699451 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:13.699946 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:13.699952 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:13.700134 (+ 182us) client_negotiation.cc:769] Sending connection context
0811 18:41:13.700275 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 18:41:13.700670 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178}
I20260811 18:41:13.702309 11910 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37311 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:41:13 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473673, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.188@KRBTEST.COM for oryx/127.30.185.188@KRBTEST.COM
I20260811 18:41:13.714993 11924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:13.703003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37311 (local address 127.0.0.1:53210)
0811 18:41:13.703175 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:13.703189 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:13.703295 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:13.703548 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:13.703551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:13.703862 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:13.703988 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:13.703995 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.705018 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.705024 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:13.706166 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:13.706179 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.706926 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.706931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:13.707387 (+ 456us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:13.707939 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:13.707956 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:13.709940 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:13.712606 (+ 2666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:13.712609 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:13.712612 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:13.713015 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:13.713454 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:13.713461 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:13.713467 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:13.713623 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:13.714162 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:13.714165 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:13.714327 (+ 162us) client_negotiation.cc:769] Sending connection context
0811 18:41:13.714441 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 18:41:13.714671 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":41}
I20260811 18:41:13.715072 11921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:13.703097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53210 (local address 127.30.185.188:37311)
0811 18:41:13.703246 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:13.703251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:13.703268 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:41:13.703387 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:13.703390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:13.703441 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:13.703513 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:13.704100 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.704867 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.706341 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.706775 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.706919 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:13.710124 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:13.710129 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:13.710133 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:13.710161 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:13.712486 (+ 2325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.713117 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.713122 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.713124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.713307 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.713719 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.713722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.713724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.714062 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:13.714172 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:13.714545 (+ 373us) server_negotiation.cc:299] Negotiation successful
0811 18:41:13.714800 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":59}
I20260811 18:41:13.719959 11910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc: Bootstrap starting.
I20260811 18:41:13.723490 11910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:13.724632 11910 log.cc:824] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc: Log is configured to *not* fsync() on all Append() calls
I20260811 18:41:13.727370 11910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc: No bootstrap required, opened a new log
I20260811 18:41:13.735370 11910 raft_consensus.cc:348] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37311 } }
I20260811 18:41:13.735813 11910 raft_consensus.cc:374] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:13.735960 11910 raft_consensus.cc:729] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eee2ef3d5cd1440eafaa7cd2f3a4fccc, State: Initialized, Role: FOLLOWER
I20260811 18:41:13.736567 11910 consensus_queue.cc:260] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37311 } }
I20260811 18:41:13.737651 11927 sys_catalog.cc:455] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37311 } } }
I20260811 18:41:13.737991 11927 sys_catalog.cc:458] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:41:13.738741 11910 sys_catalog.cc:564] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [sys.catalog]: configured and running, proceeding with master startup.
W20260811 18:41:13.748472 11943 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:41:13.748644 11943 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:41:13.752856 11612 master_runner.cc:428] Detected that this master eee2ef3d5cd1440eafaa7cd2f3a4fccc is joining an existing cluster
I20260811 18:41:13.752998 11612 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.185.188:37311
I20260811 18:41:13.772168 11927 raft_consensus.cc:482] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:41:13.772655 11927 raft_consensus.cc:504] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37311 } }
I20260811 18:41:13.776573 10544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0691bb2c526b42a697988a418b6d8750" is_pre_election: true
I20260811 18:41:13.776840 11927 leader_election.cc:288] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0691bb2c526b42a697988a418b6d8750 (127.30.185.190:39547), ef1b11a131674161bb732b9c0300aae9 (127.30.185.189:41891)
I20260811 18:41:13.776901 11689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef1b11a131674161bb732b9c0300aae9" is_pre_election: true
I20260811 18:41:13.779053 11806 leader_election.cc:302] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eee2ef3d5cd1440eafaa7cd2f3a4fccc; no voters: 0691bb2c526b42a697988a418b6d8750, ef1b11a131674161bb732b9c0300aae9
I20260811 18:41:13.779899 11927 raft_consensus.cc:2726] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:41:13.785671 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:13.758934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.0.0.1:42384)
0811 18:41:13.760875 (+ 1941us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:13.760904 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:13.761285 (+ 381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:13.762167 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:13.762175 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:13.763600 (+ 1425us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:13.764264 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:13.764285 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.766973 (+ 2688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.766983 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:13.769130 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:13.769151 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.770685 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.770695 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:13.770935 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:13.772191 (+ 1256us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:13.772215 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:13.775761 (+ 3546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:13.781672 (+ 5911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:13.781682 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:13.781687 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:13.782349 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:13.782860 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:13.782865 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:13.782874 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:13.783128 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:13.784187 (+ 1059us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:13.784193 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:13.784481 (+ 288us) client_negotiation.cc:769] Sending connection context
0811 18:41:13.784652 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 18:41:13.785082 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1601,"thread_start_us":1286,"threads_started":1}
I20260811 18:41:13.786782 11923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:13.758702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42384 (local address 127.30.185.190:39547)
0811 18:41:13.758895 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:13.758902 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:13.761028 (+ 2126us) server_negotiation.cc:407] Connection header received
0811 18:41:13.761519 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:13.761529 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:13.761704 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:13.761808 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:13.764775 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.766584 (+ 1809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.769438 (+ 2854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.770151 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.770297 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:13.776090 (+ 5793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:13.776096 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:13.776100 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:13.776137 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:13.781503 (+ 5366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.782501 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.782508 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.782512 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.782689 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.783292 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.783298 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.783302 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.783859 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:13.784092 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:13.786044 (+ 1952us) server_negotiation.cc:299] Negotiation successful
0811 18:41:13.786332 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 18:41:13.792856 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 10644
I20260811 18:41:13.794145 11812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:13.758777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43828 (local address 127.30.185.189:41891)
0811 18:41:13.759342 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:13.759348 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:13.768067 (+ 8719us) server_negotiation.cc:407] Connection header received
0811 18:41:13.768420 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:13.768427 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:13.768546 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:13.768641 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:13.771265 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.773258 (+ 1993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.779766 (+ 6508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.780475 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.780579 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:13.783657 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:13.783664 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:13.783669 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:13.783712 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:13.788630 (+ 4918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.791272 (+ 2642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.791278 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.791282 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.791496 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.792061 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.792069 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.792075 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.792465 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:13.792629 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:13.793491 (+ 862us) server_negotiation.cc:299] Negotiation successful
0811 18:41:13.793753 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291}
I20260811 18:41:13.795620 11953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:13.759335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:41891 (local address 127.0.0.1:43828)
0811 18:41:13.767941 (+ 8606us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:13.767958 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:13.768155 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:13.768900 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:13.768906 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:13.770095 (+ 1189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:13.771007 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:13.771039 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.774716 (+ 3677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.774722 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:13.779574 (+ 4852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:13.779588 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.780975 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.780984 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:13.781532 (+ 548us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:13.782211 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:13.782235 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:13.783443 (+ 1208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:13.790486 (+ 7043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:13.790492 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:13.790496 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:13.791101 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:13.791663 (+ 562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:13.791669 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:13.791673 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:13.791897 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:13.792711 (+ 814us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:13.792716 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:13.793124 (+ 408us) client_negotiation.cc:769] Sending connection context
0811 18:41:13.794465 (+ 1341us) client_negotiation.cc:240] Negotiation successful
0811 18:41:13.794908 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8430,"thread_start_us":8113,"threads_started":1}
I20260811 18:41:13.801811 11955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:13.760756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37311 (local address 127.0.0.1:53224)
0811 18:41:13.768702 (+ 7946us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:13.768774 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:13.768963 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:13.769637 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:13.769643 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:13.771099 (+ 1456us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:13.771781 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:13.771821 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.775009 (+ 3188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.775014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:13.782886 (+ 7872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:13.782899 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.785113 (+ 2214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.785123 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:13.785373 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:13.786615 (+ 1242us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:13.786640 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:13.788233 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:13.797873 (+ 9640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:13.797882 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:13.797889 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:13.798851 (+ 962us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:13.799414 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:13.799422 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:13.799428 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:13.799611 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:13.800319 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:13.800324 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:13.800556 (+ 232us) client_negotiation.cc:769] Sending connection context
0811 18:41:13.800763 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 18:41:13.801150 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7745,"thread_start_us":6291,"threads_started":1}
I20260811 18:41:13.804296 11921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:13.761277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53224 (local address 127.30.185.188:37311)
0811 18:41:13.761517 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:13.761526 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:13.768843 (+ 7317us) server_negotiation.cc:407] Connection header received
0811 18:41:13.769137 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:13.769145 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:13.769267 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:13.769339 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:13.772462 (+ 3123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.773703 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.783063 (+ 9360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.784296 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.784418 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:13.788420 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:13.788427 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:13.788432 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:13.788474 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:13.797667 (+ 9193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.799029 (+ 1362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.799037 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.799043 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.799266 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.799752 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.799757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.799760 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.800154 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:13.800338 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:13.801410 (+ 1072us) server_negotiation.cc:299] Negotiation successful
0811 18:41:13.801663 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36,"spinlock_wait_cycles":26496}
Aug 11 18:41:13 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473670, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.190@KRBTEST.COM for oryx/127.30.185.188@KRBTEST.COM
I20260811 18:41:13.821396 11958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:13.794913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53234 (local address 127.30.185.188:37311)
0811 18:41:13.800165 (+ 5252us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:13.800174 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:13.800194 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:41:13.800334 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:13.800339 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:13.800436 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:13.800580 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:13.801647 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.805320 (+ 3673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.807124 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.807874 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.809134 (+ 1260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:13.813260 (+ 4126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:13.813268 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:13.813273 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:13.813314 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:13.817502 (+ 4188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.818433 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.818440 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.818443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.818661 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.819181 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.819186 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.819190 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.819639 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:13.819793 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:13.820573 (+ 780us) server_negotiation.cc:299] Negotiation successful
0811 18:41:13.820817 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5071,"thread_start_us":4502,"threads_started":1}
I20260811 18:41:13.821391 11808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:13.794964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37311 (local address 127.0.0.1:53234)
0811 18:41:13.795418 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:13.795445 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:13.795593 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:13.800642 (+ 5049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:13.800649 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:13.801133 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:13.801431 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:13.801445 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.805493 (+ 4048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.805499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:13.806907 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:13.806936 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.808067 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.808074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:13.808409 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:13.809474 (+ 1065us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:13.809500 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:13.812969 (+ 3469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:13.817652 (+ 4683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:13.817658 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:13.817662 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:13.818277 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:13.818824 (+ 547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:13.818830 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:13.818834 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:13.819023 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:13.819963 (+ 940us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:13.819968 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:13.820203 (+ 235us) client_negotiation.cc:769] Sending connection context
0811 18:41:13.820438 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 18:41:13.820777 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201}
I20260811 18:41:13.823074 10517 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.185.188:37311: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc"
member_type: NON_VOTER
last_known_addr {
host: "127.30.185.188"
port: 37311
}
attrs {
promote: true
}
}
dest_uuid: "0691bb2c526b42a697988a418b6d8750"
cas_config_opid_index: 5
Aug 11 18:41:13 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.185.129@KRBTEST.COM
I20260811 18:41:13.824291 10517 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: NON_VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: true } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: NON_VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: true } }
I20260811 18:41:13.826642 11807 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER eee2ef3d5cd1440eafaa7cd2f3a4fccc (127.30.185.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: NON_VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: true } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: NON_VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: true } } }
I20260811 18:41:13.827333 11689 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 1 LEARNER]: Refusing update from remote peer 0691bb2c526b42a697988a418b6d8750: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 18:41:13.832458 11859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:13.834546 11959 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: NON_VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 18:41:13.836220 11957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:13.794081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33182 (local address 127.30.185.129:34579)
0811 18:41:13.810106 (+ 16025us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:13.810116 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:13.810147 (+ 31us) server_negotiation.cc:407] Connection header received
0811 18:41:13.810217 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:13.810227 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:13.810517 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:13.812762 (+ 2245us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:13.812989 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.814870 (+ 1881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.817460 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.818171 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.818318 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:13.825364 (+ 7046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:13.825379 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:13.825421 (+ 42us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:13.825466 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:13.830837 (+ 5371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.831941 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.831950 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.831954 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.832183 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.832921 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.832925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.832928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.833619 (+ 691us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:13.835193 (+ 1574us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:13.835362 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 18:41:13.835647 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15854,"thread_start_us":15460,"threads_started":1}
I20260811 18:41:13.837381 11859 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 1 FOLLOWER]: Refusing update from remote peer 0691bb2c526b42a697988a418b6d8750: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 18:41:13.839664 10872 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:13.845611 11807 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.185.188:37311
I20260811 18:41:13.845728 11962 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: NON_VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:41:13.846585 11612 master.cc:537] Master@127.30.185.188:37311 shutting down...
I20260811 18:41:13.847824 11807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0691bb2c526b42a697988a418b6d8750" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: NON_VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: true } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: NON_VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: true } } }
I20260811 18:41:13.848150 11807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:13.857934 11689 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER eee2ef3d5cd1440eafaa7cd2f3a4fccc (127.30.185.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: NON_VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: true } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: NON_VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: true } } }
W20260811 18:41:13.862229 10486 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 -> Peer eee2ef3d5cd1440eafaa7cd2f3a4fccc (127.30.185.188:37311): Couldn't send request to peer eee2ef3d5cd1440eafaa7cd2f3a4fccc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:41:13.863909 11741 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0691bb2c526b42a697988a418b6d8750" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: NON_VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: true } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: NON_VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: true } } }
I20260811 18:41:13.864246 11741 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [sys.catalog]: This master's current role is: LEARNER
I20260811 18:41:13.865350 11962 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750: attempting to promote NON_VOTER ef1b11a131674161bb732b9c0300aae9 to VOTER
I20260811 18:41:13.866142 11962 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: NON_VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: true } }
I20260811 18:41:13.868502 11689 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 1 LEARNER]: Refusing update from remote peer 0691bb2c526b42a697988a418b6d8750: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 18:41:13.869118 11962 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260811 18:41:13.869467 10486 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 -> Peer eee2ef3d5cd1440eafaa7cd2f3a4fccc (127.30.185.188:37311): Couldn't send request to peer eee2ef3d5cd1440eafaa7cd2f3a4fccc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:41:13.871637 11807 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ef1b11a131674161bb732b9c0300aae9 (127.30.185.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: NON_VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: true } } }
I20260811 18:41:13.874035 11689 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ef1b11a131674161bb732b9c0300aae9 (127.30.185.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: NON_VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: true } } }
I20260811 18:41:13.876096 11959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0691bb2c526b42a697988a418b6d8750" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: NON_VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: true } } }
I20260811 18:41:13.876475 11959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:13.876860 11959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0691bb2c526b42a697988a418b6d8750" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: NON_VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: true } } }
I20260811 18:41:13.877135 11959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:13.879467 11741 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0691bb2c526b42a697988a418b6d8750" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: NON_VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: true } } }
I20260811 18:41:13.879714 11741 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 18:41:13.879779 11821 tablet.cc:2369] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:41:13.884261 11612 raft_consensus.cc:2229] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:41:13.884682 11612 raft_consensus.cc:2258] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:41:13.884968 11612 tablet_replica.cc:330] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc: stopping tablet replica
I20260811 18:41:13.891829 11612 master.cc:552] Master@127.30.185.188:37311 shutdown complete.
I20260811 18:41:13.891939 11612 master_runner.cc:305] Clearing existing system tablet
I20260811 18:41:13.892303 11612 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:13.892549 11612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:13.895541 11612 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260811 18:41:13.897024 11970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:13.897430 11612 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:13.897733 11612 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:41:13.897835 11612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/wal
uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc"
format_stamp: "Formatted at 2026-08-11 18:41:13 on dist-test-slave-z8x9"
server_key: "88e80bc6055ef657f697331ca3722eb2"
server_key_iv: "72391819504e33c90d3bc357cd07433c"
server_key_version: "encryptionkey@0"
I20260811 18:41:13.901108 11612 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:41:13.906672 11612 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 18:41:13.906980 11612 log.cc:1196] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 18:41:13.907457 11612 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc: Deleting consensus metadata
I20260811 18:41:13.907753 11612 master_runner.cc:315] Copying system tablet from 127.30.185.190:39547
I20260811 18:41:13.910409 11612 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc: tablet copy: Beginning tablet copy session from remote peer at address 127.30.185.190:39547
I20260811 18:41:13.924947 11923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:13.911251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42398 (local address 127.30.185.190:39547)
0811 18:41:13.911510 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:13.911518 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:13.912081 (+ 563us) server_negotiation.cc:407] Connection header received
0811 18:41:13.912327 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:13.912332 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:13.912412 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:13.912524 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:13.913634 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.914628 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.916053 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.916506 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.916656 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:13.918702 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:13.918709 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:13.918715 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:13.918754 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:13.921981 (+ 3227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.922770 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.922774 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.922777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.922903 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:13.923377 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:13.923382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:13.923386 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:13.923689 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:13.923849 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:13.924330 (+ 481us) server_negotiation.cc:299] Negotiation successful
0811 18:41:13.924569 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260811 18:41:13.925331 11981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:13.911225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.0.0.1:42398)
0811 18:41:13.911958 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:13.911980 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:13.912164 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:13.912556 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:13.912560 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:13.913108 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:13.913478 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:13.913491 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.914772 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.914778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:13.915902 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:13.915915 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:13.916679 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:13.916686 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:13.916939 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:13.917728 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:13.917746 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:13.918493 (+ 747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:13.922158 (+ 3665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:13.922164 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:13.922175 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:13.922541 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:13.923026 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:13.923032 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:13.923035 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:13.923250 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:13.923814 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:13.923820 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:13.923991 (+ 171us) client_negotiation.cc:769] Sending connection context
0811 18:41:13.924142 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 18:41:13.924514 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":462,"threads_started":1}
I20260811 18:41:13.926127 10564 tablet_copy_service.cc:140] P 0691bb2c526b42a697988a418b6d8750: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer eee2ef3d5cd1440eafaa7cd2f3a4fccc ({username='oryx', principal='oryx/127.30.185.188@KRBTEST.COM'} at 127.0.0.1:42398)
I20260811 18:41:13.926362 10564 tablet_copy_service.cc:161] P 0691bb2c526b42a697988a418b6d8750: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer eee2ef3d5cd1440eafaa7cd2f3a4fccc at {username='oryx', principal='oryx/127.30.185.188@KRBTEST.COM'} at 127.0.0.1:42398: session id = eee2ef3d5cd1440eafaa7cd2f3a4fccc-00000000000000000000000000000000
I20260811 18:41:13.928803 10564 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 18:41:13.930594 11612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 18:41:13.936793 11612 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc: tablet copy: Starting download of 0 data blocks...
I20260811 18:41:13.937206 11612 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc: tablet copy: Starting download of 1 WAL segments...
I20260811 18:41:13.939807 11612 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 18:41:13.943783 10564 tablet_copy_service.cc:342] P 0691bb2c526b42a697988a418b6d8750: Request end of tablet copy session eee2ef3d5cd1440eafaa7cd2f3a4fccc-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.185.188@KRBTEST.COM'} at 127.0.0.1:42398
I20260811 18:41:13.943938 10564 tablet_copy_service.cc:433] P 0691bb2c526b42a697988a418b6d8750: ending tablet copy session eee2ef3d5cd1440eafaa7cd2f3a4fccc-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer eee2ef3d5cd1440eafaa7cd2f3a4fccc
I20260811 18:41:13.957032 11612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:13.960460 11989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:13.960623 11985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:13.965503 11992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:13.972263 11612 server_base.cc:1034] running on GCE node
I20260811 18:41:13.972702 11612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:13.973625 11612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:13.974874 11612 hybrid_clock.cc:648] HybridClock initialized: now 1786473673974814 us; error 38 us; skew 500 ppm
I20260811 18:41:13.975255 11612 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:13 dist-test-slave-z8x9 krb5kdc[10417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473673, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:13.978003 11612 init.cc:377] Logged in from keytab as oryx/127.30.185.188@KRBTEST.COM (short username oryx)
I20260811 18:41:13.982437 11612 webserver.cc:460] Webserver started at http://127.30.185.188:46473/ using document root <none> and password file <none>
I20260811 18:41:13.982846 11612 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:13.983117 11612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:13.987040 11612 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 18:41:13.989598 12010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:13.990077 11612 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260811 18:41:13.990279 11612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/wal
uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc"
format_stamp: "Formatted at 2026-08-11 18:41:13 on dist-test-slave-z8x9"
server_key: "88e80bc6055ef657f697331ca3722eb2"
server_key_iv: "72391819504e33c90d3bc357cd07433c"
server_key_version: "encryptionkey@0"
I20260811 18:41:13.990561 11612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:13.996311 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 10644
I20260811 18:41:14.022066 11612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:14.026309 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:34579
--local_ip_for_outbound_sockets=127.30.185.129
--tserver_master_addrs=127.30.185.190:39547
--webserver_port=42791
--webserver_interface=127.30.185.129
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.129.keytab
--principal=oryx/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.185.190:39547,127.30.185.189:41891
--tserver_master_addrs=127.30.185.190:39547,127.30.185.189:41891,127.30.185.188:37311 with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:41:14.026569 11612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:14.065402 11612 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.188:37311
I20260811 18:41:14.065428 12127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.188:37311 every 8 connection(s)
I20260811 18:41:14.066371 11612 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/master-2/data/info.pb
I20260811 18:41:14.073707 12128 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:41:14.079175 12128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc: Bootstrap starting.
I20260811 18:41:14.098493 12128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:41:14.099095 12128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc: Bootstrap complete.
I20260811 18:41:14.100018 12128 raft_consensus.cc:348] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: NON_VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: true } }
I20260811 18:41:14.100335 12128 raft_consensus.cc:729] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eee2ef3d5cd1440eafaa7cd2f3a4fccc, State: Initialized, Role: LEARNER
I20260811 18:41:14.100603 12128 consensus_queue.cc:260] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: NON_VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: true } }
I20260811 18:41:14.101671 12137 sys_catalog.cc:455] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: NON_VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: true } } }
I20260811 18:41:14.102067 12128 sys_catalog.cc:564] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:41:14.102073 12137 sys_catalog.cc:458] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [sys.catalog]: This master's current role is: LEARNER
I20260811 18:41:14.118247 12165 catalog_manager.cc:1259] Loaded cluster ID: 6cdce79fc4a64402b260ed180c0557fb
I20260811 18:41:14.118376 12165 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc: loading cluster ID for follower catalog manager: success
I20260811 18:41:14.120803 12165 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc: acquiring CA information for follower catalog manager: success
I20260811 18:41:14.123003 12165 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:41:14.325026 11766 catalog_manager.cc:1259] Loaded cluster ID: 6cdce79fc4a64402b260ed180c0557fb
I20260811 18:41:14.325150 11766 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9: loading cluster ID for follower catalog manager: success
I20260811 18:41:14.328920 11766 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9: acquiring CA information for follower catalog manager: success
I20260811 18:41:14.331560 11766 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:41:14.440397 12169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:14.432618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53250 (local address 127.30.185.188:37311)
0811 18:41:14.433399 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:14.433404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:14.433422 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:41:14.433487 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:14.433490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:14.433559 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:14.433662 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:14.434242 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:14.435276 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:14.438477 (+ 3201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:14.439335 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:14.439497 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:14.439638 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:14.439965 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 18:41:14.440070 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":490,"threads_started":1}
I20260811 18:41:14.440441 12167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:14.432280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37311 (local address 127.0.0.1:53250)
0811 18:41:14.433035 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:14.433055 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:14.433255 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:14.433757 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:14.433762 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:14.433795 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:14.434033 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:14.434056 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:14.435445 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:14.435451 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:14.438284 (+ 2833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:14.438304 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:14.439518 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:14.439525 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:14.439749 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:14.439818 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 18:41:14.439928 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 18:41:14.440078 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":476,"threads_started":1}
I20260811 18:41:14.441661 12069 raft_consensus.cc:1203] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 18:41:14.442322 12137 sys_catalog.cc:455] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0691bb2c526b42a697988a418b6d8750. Latest consensus state: current_term: 1 leader_uuid: "0691bb2c526b42a697988a418b6d8750" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: NON_VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: true } } }
I20260811 18:41:14.442667 12137 sys_catalog.cc:458] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [sys.catalog]: This master's current role is: LEARNER
W20260811 18:41:14.467000 12043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:14.467376 12043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:14.467501 12043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:14.476819 12043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:14.476994 12043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:14.477119 12043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:41:14.488536 12043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.129.keytab
--principal=oryx/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:34579
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=42791
--tserver_master_addrs=127.30.185.190:39547,127.30.185.189:41891,127.30.185.188:37311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:14.490229 12043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:14.491777 12043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:14.507136 12179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:14.507202 12177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:14.508167 12182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:14.508512 12043 server_base.cc:1034] running on GCE node
I20260811 18:41:14.509145 12043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:14.510161 12043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:14.511389 12043 hybrid_clock.cc:648] HybridClock initialized: now 1786473674511335 us; error 42 us; skew 500 ppm
I20260811 18:41:14.511740 12043 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:14 dist-test-slave-z8x9 krb5kdc[10417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473674, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:14.515583 12043 init.cc:377] Logged in from keytab as oryx/127.30.185.129@KRBTEST.COM (short username oryx)
I20260811 18:41:14.517153 12043 webserver.cc:460] Webserver started at http://127.30.185.129:42791/ using document root <none> and password file <none>
I20260811 18:41:14.517999 12043 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:14.518170 12043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:14.524003 12043 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 18:41:14.527117 12195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:14.528477 12043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:41:14.528671 12043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "b92d06495a8d4bd1a82526fe9071c9d1"
format_stamp: "Formatted at 2026-08-11 18:41:11 on dist-test-slave-z8x9"
server_key: "eeff0dc1c4748c3ea6b6b2130f7eb15d"
server_key_iv: "fb078b554ff8b12c5059520a99056ee9"
server_key_version: "encryptionkey@0"
I20260811 18:41:14.529974 12043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:14.546650 12043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:14.551059 12043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:14.551374 12043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:14.552531 12043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:14.554399 12043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:14.554531 12043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:14.554668 12043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:14.554762 12043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:14.604238 12043 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:34579
I20260811 18:41:14.604346 12408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:34579 every 8 connection(s)
I20260811 18:41:14.606122 12043 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:41:14.613000 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 12043
I20260811 18:41:14.613457 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 10890
Aug 11 18:41:14 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.185.130@KRBTEST.COM
Aug 11 18:41:14 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473674, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.188@KRBTEST.COM
Aug 11 18:41:14 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473674, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM
Aug 11 18:41:14 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473674, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.189@KRBTEST.COM
I20260811 18:41:14.649609 12169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:14.622972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:53819 (local address 127.30.185.188:37311)
0811 18:41:14.623200 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:14.623207 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:14.625555 (+ 2348us) server_negotiation.cc:407] Connection header received
0811 18:41:14.626723 (+ 1168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:14.626730 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:14.626820 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:14.626903 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:14.630467 (+ 3564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:14.631518 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:14.635705 (+ 4187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:14.636392 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:14.636497 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:14.641634 (+ 5137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:14.641640 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:14.641643 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:14.641706 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:14.645493 (+ 3787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:14.646549 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:14.646554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:14.646558 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:14.646715 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:14.647165 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:14.647170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:14.647174 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:14.647581 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:14.647761 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:14.648889 (+ 1128us) server_negotiation.cc:299] Negotiation successful
0811 18:41:14.649171 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 18:41:14.650894 12427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:14.611619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:41891 (local address 127.30.185.129:60771)
0811 18:41:14.622026 (+ 10407us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:14.622215 (+ 189us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:14.626361 (+ 4146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:14.627437 (+ 1076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:14.627449 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:14.628981 (+ 1532us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:14.630259 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:14.630268 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:14.631902 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:14.631908 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:14.634843 (+ 2935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:14.634859 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:14.636401 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:14.636409 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:14.636629 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:14.637524 (+ 895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:14.637553 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:14.642759 (+ 5206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:14.647282 (+ 4523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:14.647287 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:14.647290 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:14.647839 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:14.648250 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:14.648255 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:14.648259 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:14.648503 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:14.649795 (+ 1292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:14.649800 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:14.649969 (+ 169us) client_negotiation.cc:769] Sending connection context
0811 18:41:14.650093 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 18:41:14.650290 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10107,"mutex_wait_us":656,"spinlock_wait_cycles":65536,"thread_start_us":10411,"threads_started":1}
I20260811 18:41:14.650944 12437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:14.623312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37311 (local address 127.30.185.129:53819)
0811 18:41:14.625429 (+ 2117us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:14.625448 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:14.626387 (+ 939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:14.627440 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:14.627450 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:14.628941 (+ 1491us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:14.630218 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:14.630250 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:14.631740 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:14.631745 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:14.635517 (+ 3772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:14.635532 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:14.636874 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:14.636883 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:14.637104 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:14.637663 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:14.637681 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:14.641246 (+ 3565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:14.645678 (+ 4432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:14.645688 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:14.645707 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:14.646381 (+ 674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:14.646822 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:14.646827 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:14.646838 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:14.647036 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:14.647751 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:14.647760 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:14.648263 (+ 503us) client_negotiation.cc:769] Sending connection context
0811 18:41:14.648721 (+ 458us) client_negotiation.cc:240] Negotiation successful
0811 18:41:14.649330 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1990,"mutex_wait_us":222,"spinlock_wait_cycles":4480,"thread_start_us":1275,"threads_started":1}
I20260811 18:41:14.652088 12420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:14.610143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:60771 (local address 127.30.185.189:41891)
0811 18:41:14.625113 (+ 14970us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:14.625123 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:14.625152 (+ 29us) server_negotiation.cc:407] Connection header received
0811 18:41:14.626814 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:14.626819 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:14.626944 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:14.627879 (+ 935us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:14.630577 (+ 2698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:14.631721 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:14.635207 (+ 3486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:14.636134 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:14.636718 (+ 584us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:14.643055 (+ 6337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:14.643065 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:14.643073 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:14.643128 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:14.647145 (+ 4017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:14.647980 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:14.647986 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:14.647990 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:14.648128 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:14.648634 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:14.648639 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:14.648642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:14.649194 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:14.651177 (+ 1983us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:14.651327 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 18:41:14.651611 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14838,"thread_start_us":14668,"threads_started":1}
I20260811 18:41:14.652956 12411 heartbeater.cc:347] Connected to a master server at 127.30.185.188:37311
I20260811 18:41:14.653355 12411 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:14.653864 12424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:14.610853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.30.185.129:54169)
0811 18:41:14.622026 (+ 11173us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:14.622519 (+ 493us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:14.626361 (+ 3842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:14.627437 (+ 1076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:14.627449 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:14.628928 (+ 1479us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:14.630218 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:14.630250 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:14.631932 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:14.631938 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:14.634802 (+ 2864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:14.634840 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:14.636211 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:14.636219 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:14.636572 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:14.638178 (+ 1606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:14.638222 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:14.641586 (+ 3364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:14.648675 (+ 7089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:14.648682 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:14.648685 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:14.649254 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:14.650638 (+ 1384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:14.650643 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:14.650646 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:14.651364 (+ 718us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:14.652508 (+ 1144us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:14.652513 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:14.652756 (+ 243us) client_negotiation.cc:769] Sending connection context
0811 18:41:14.652906 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 18:41:14.653270 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10755,"mutex_wait_us":348,"spinlock_wait_cycles":31488,"thread_start_us":10457,"threads_started":1}
I20260811 18:41:14.653869 12421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:14.610277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:54169 (local address 127.30.185.190:39547)
0811 18:41:14.617680 (+ 7403us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:14.617685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:14.622606 (+ 4921us) server_negotiation.cc:407] Connection header received
0811 18:41:14.626711 (+ 4105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:14.626718 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:14.626821 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:14.626908 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:14.630466 (+ 3558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:14.631699 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:14.635050 (+ 3351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:14.635752 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:14.635849 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:14.641867 (+ 6018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:14.641874 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:14.641879 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:14.641921 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:14.646285 (+ 4364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:14.649388 (+ 3103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:14.649395 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:14.649398 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:14.649635 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:14.651511 (+ 1876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:14.651520 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:14.651526 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:14.652374 (+ 848us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:14.652557 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:14.653069 (+ 512us) server_negotiation.cc:299] Negotiation successful
0811 18:41:14.653370 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7275,"thread_start_us":7125,"threads_started":1}
I20260811 18:41:14.654484 12411 heartbeater.cc:511] Master 127.30.185.188:37311 requested a full tablet report, sending...
I20260811 18:41:14.655031 12416 heartbeater.cc:347] Connected to a master server at 127.30.185.190:39547
I20260811 18:41:14.655180 12416 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:14.655493 12432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:14.615208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35722 (local address 127.30.185.130:44261)
0811 18:41:14.616087 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:14.616100 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:14.624547 (+ 8447us) server_negotiation.cc:407] Connection header received
0811 18:41:14.624762 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:14.624775 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:14.625065 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:14.625274 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:14.627223 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:14.628969 (+ 1746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:14.631425 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:14.632599 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:14.632786 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:14.642220 (+ 9434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:14.642238 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:14.642257 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:14.642299 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:14.647620 (+ 5321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:14.649456 (+ 1836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:14.649465 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:14.649468 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:14.650041 (+ 573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:14.651565 (+ 1524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:14.651572 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:14.651577 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:14.652490 (+ 913us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:14.652856 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:14.654530 (+ 1674us) server_negotiation.cc:299] Negotiation successful
0811 18:41:14.654961 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":483,"threads_started":1}
I20260811 18:41:14.655710 12416 heartbeater.cc:511] Master 127.30.185.190:39547 requested a full tablet report, sending...
I20260811 18:41:14.656281 12417 heartbeater.cc:347] Connected to a master server at 127.30.185.189:41891
I20260811 18:41:14.656399 12417 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:14.656961 12417 heartbeater.cc:511] Master 127.30.185.189:41891 requested a full tablet report, sending...
I20260811 18:41:14.657529 12050 ts_manager.cc:194] Registered new tserver with Master: b92d06495a8d4bd1a82526fe9071c9d1 (127.30.185.129:34579)
I20260811 18:41:14.657609 11124 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:14.658841 10517 ts_manager.cc:194] Re-registered known tserver with Master: b92d06495a8d4bd1a82526fe9071c9d1 (127.30.185.129:34579)
I20260811 18:41:14.659178 11667 ts_manager.cc:194] Registered new tserver with Master: b92d06495a8d4bd1a82526fe9071c9d1 (127.30.185.129:34579)
I20260811 18:41:14.660121 10517 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:54169
I20260811 18:41:14.753988 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 10890
I20260811 18:41:14.783711 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:44261
--local_ip_for_outbound_sockets=127.30.185.130
--tserver_master_addrs=127.30.185.190:39547
--webserver_port=40637
--webserver_interface=127.30.185.130
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.130.keytab
--principal=oryx/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.185.190:39547,127.30.185.189:41891
--tserver_master_addrs=127.30.185.190:39547,127.30.185.189:41891,127.30.185.188:37311 with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:41:14.913864 11807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0691bb2c526b42a697988a418b6d8750" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: NON_VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: true } } }
I20260811 18:41:14.914254 11807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:14.916374 12444 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750: attempting to promote NON_VOTER eee2ef3d5cd1440eafaa7cd2f3a4fccc to VOTER
I20260811 18:41:14.917805 12444 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: false } }
I20260811 18:41:14.920313 12069 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 1 LEARNER]: Refusing update from remote peer 0691bb2c526b42a697988a418b6d8750: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 18:41:14.920421 11689 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 1 FOLLOWER]: Refusing update from remote peer 0691bb2c526b42a697988a418b6d8750: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 18:41:14.921430 12444 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260811 18:41:14.921974 11807 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 18:41:14.925359 12442 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, eee2ef3d5cd1440eafaa7cd2f3a4fccc (127.30.185.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: false } } }
I20260811 18:41:14.926137 11689 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, eee2ef3d5cd1440eafaa7cd2f3a4fccc (127.30.185.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: false } } }
I20260811 18:41:14.929636 12069 raft_consensus.cc:2932] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, eee2ef3d5cd1440eafaa7cd2f3a4fccc (127.30.185.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: false } } }
I20260811 18:41:14.930081 12442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0691bb2c526b42a697988a418b6d8750" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: false } } }
I20260811 18:41:14.930740 12442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:14.931013 12442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0691bb2c526b42a697988a418b6d8750" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: false } } }
I20260811 18:41:14.931576 12442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:14.934733 11807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0691bb2c526b42a697988a418b6d8750" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: false } } }
I20260811 18:41:14.935029 11807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:14.939770 12137 sys_catalog.cc:455] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0691bb2c526b42a697988a418b6d8750" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: false } } }
I20260811 18:41:14.940105 12137 sys_catalog.cc:458] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:41:14.945779 12454 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0691bb2c526b42a697988a418b6d8750" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: false } } }
I20260811 18:41:14.946395 12454 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 18:41:15.151319 12440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:15.151681 12440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:15.151819 12440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:15.159627 12440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:15.159786 12440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:15.159966 12440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:41:15.170271 12440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.130.keytab
--principal=oryx/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:44261
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=40637
--tserver_master_addrs=127.30.185.190:39547,127.30.185.189:41891,127.30.185.188:37311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:15.171738 12440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:15.173518 12440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:15.187670 12469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:15.187649 12470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:15.187634 12474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:15.190796 12440 server_base.cc:1034] running on GCE node
I20260811 18:41:15.191555 12440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:15.192762 12440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:15.193989 12440 hybrid_clock.cc:648] HybridClock initialized: now 1786473675193936 us; error 41 us; skew 500 ppm
I20260811 18:41:15.194315 12440 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:15 dist-test-slave-z8x9 krb5kdc[10417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473675, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:15.197655 12440 init.cc:377] Logged in from keytab as oryx/127.30.185.130@KRBTEST.COM (short username oryx)
I20260811 18:41:15.199229 12440 webserver.cc:460] Webserver started at http://127.30.185.130:40637/ using document root <none> and password file <none>
I20260811 18:41:15.199971 12440 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:15.200146 12440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:15.205902 12440 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:41:15.208752 12484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:15.210122 12440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:41:15.210306 12440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "ce2d62517cd247bd98c6e31ea0f88cc8"
format_stamp: "Formatted at 2026-08-11 18:41:11 on dist-test-slave-z8x9"
server_key: "73bf429f0111a93afa373efd098263c9"
server_key_iv: "3b014e08e6f1fdbf1f7c6026e46f4c7b"
server_key_version: "encryptionkey@0"
I20260811 18:41:15.211290 12440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:15.222644 12440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:15.226662 12440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:15.226975 12440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:15.227985 12440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:15.229467 12440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:15.229588 12440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:15.229714 12440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:15.229799 12440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:15.265023 12440 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:44261
I20260811 18:41:15.265110 12725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:44261 every 8 connection(s)
I20260811 18:41:15.266598 12440 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:41:15.274691 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 12440
I20260811 18:41:15.274995 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 11147
Aug 11 18:41:15 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.185.131@KRBTEST.COM
Aug 11 18:41:15 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473675, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.188@KRBTEST.COM
Aug 11 18:41:15 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473675, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM
Aug 11 18:41:15 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473675, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.189@KRBTEST.COM
I20260811 18:41:15.325143 12739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:15.275948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51576 (local address 127.30.185.131:42453)
0811 18:41:15.299484 (+ 23536us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:15.299499 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:15.299541 (+ 42us) server_negotiation.cc:407] Connection header received
0811 18:41:15.299645 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:15.299659 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:15.300022 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:15.300245 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:15.302285 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.304157 (+ 1872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.306218 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.307621 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.307751 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:15.315572 (+ 7821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:15.315594 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:15.315613 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:15.315670 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:15.322094 (+ 6424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:15.322850 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:15.322858 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:15.322860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:15.322988 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:15.323426 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:15.323430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:15.323432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:15.323757 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:15.324077 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:15.324386 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 18:41:15.324653 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23285,"thread_start_us":24561,"threads_started":1}
I20260811 18:41:15.326659 11404 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:15.331928 12737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:15.275734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:38843 (local address 127.30.185.188:37311)
0811 18:41:15.296186 (+ 20452us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:15.296196 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:15.297447 (+ 1251us) server_negotiation.cc:407] Connection header received
0811 18:41:15.303238 (+ 5791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:15.303242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:15.303343 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:15.303457 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:15.306922 (+ 3465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.309923 (+ 3001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.313278 (+ 3355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.314066 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.314197 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:15.320629 (+ 6432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:15.320636 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:15.320641 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:15.320682 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:15.324756 (+ 4074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:15.325931 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:15.325938 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:15.325943 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:15.326138 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:15.326729 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:15.326734 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:15.326737 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:15.327193 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:15.327378 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:15.331229 (+ 3851us) server_negotiation.cc:299] Negotiation successful
0811 18:41:15.331420 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20275,"thread_start_us":20477,"threads_started":1}
I20260811 18:41:15.332119 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:15.274686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37311 (local address 127.30.185.130:38843)
0811 18:41:15.297197 (+ 22511us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:15.297220 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:15.302668 (+ 5448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:15.303888 (+ 1220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:15.303892 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:15.305463 (+ 1571us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:15.306695 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:15.306715 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.310131 (+ 3416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.310135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:15.313034 (+ 2899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:15.313048 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.314689 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.314700 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:15.315078 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:15.315786 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:15.315807 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:15.319310 (+ 3503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:15.324920 (+ 5610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:15.324928 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:15.324945 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:15.325760 (+ 815us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:15.326289 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:15.326294 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:15.326299 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:15.326572 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:15.327488 (+ 916us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:15.327500 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:15.328046 (+ 546us) client_negotiation.cc:769] Sending connection context
0811 18:41:15.328422 (+ 376us) client_negotiation.cc:240] Negotiation successful
0811 18:41:15.329101 (+ 679us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22303,"mutex_wait_us":205,"spinlock_wait_cycles":33920,"thread_start_us":21950,"threads_started":1}
I20260811 18:41:15.333663 12733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:15.270165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.30.185.130:49723)
0811 18:41:15.274205 (+ 4040us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:15.274259 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:15.303108 (+ 28849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:15.304043 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:15.304049 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:15.305463 (+ 1414us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:15.306683 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:15.306715 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.309049 (+ 2334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.309056 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:15.311757 (+ 2701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:15.311775 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.313153 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.313205 (+ 52us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:15.313489 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:15.314695 (+ 1206us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:15.314720 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:15.319316 (+ 4596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:15.324981 (+ 5665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:15.324986 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:15.324990 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:15.325863 (+ 873us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:15.330849 (+ 4986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:15.330856 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:15.330866 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:15.331081 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:15.332283 (+ 1202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:15.332288 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:15.332534 (+ 246us) client_negotiation.cc:769] Sending connection context
0811 18:41:15.332769 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 18:41:15.333110 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3644,"mutex_wait_us":279,"spinlock_wait_cycles":4480,"thread_start_us":3375,"threads_started":1}
I20260811 18:41:15.334102 12744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:15.291381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:52199 (local address 127.30.185.189:41891)
0811 18:41:15.298536 (+ 7155us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:15.298546 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:15.298573 (+ 27us) server_negotiation.cc:407] Connection header received
0811 18:41:15.303063 (+ 4490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:15.303070 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:15.303181 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:15.303341 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:15.307314 (+ 3973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.308621 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.311808 (+ 3187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.312457 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.312595 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:15.324884 (+ 12289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:15.324892 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:15.324896 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:15.324941 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:15.328482 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:15.330110 (+ 1628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:15.330120 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:15.330125 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:15.330360 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:15.331059 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:15.331066 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:15.331069 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:15.331437 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:15.331573 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:15.333350 (+ 1777us) server_negotiation.cc:299] Negotiation successful
0811 18:41:15.333594 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6993,"thread_start_us":6798,"threads_started":1}
I20260811 18:41:15.334065 12726 heartbeater.cc:347] Connected to a master server at 127.30.185.188:37311
I20260811 18:41:15.334514 12726 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:15.334847 12732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:15.269611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:49723 (local address 127.30.185.190:39547)
0811 18:41:15.302679 (+ 33068us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:15.302685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:15.302707 (+ 22us) server_negotiation.cc:407] Connection header received
0811 18:41:15.303550 (+ 843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:15.303572 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:15.303677 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:15.303780 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:15.307719 (+ 3939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.308884 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.312085 (+ 3201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.312729 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.312855 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:15.319631 (+ 6776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:15.319638 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:15.319644 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:15.319697 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:15.323779 (+ 4082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:15.328685 (+ 4906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:15.328694 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:15.328700 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:15.328918 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:15.331650 (+ 2732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:15.331656 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:15.331659 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:15.332024 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:15.332198 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:15.332911 (+ 713us) server_negotiation.cc:299] Negotiation successful
0811 18:41:15.334376 (+ 1465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32938,"thread_start_us":32753,"threads_started":1}
I20260811 18:41:15.335563 12726 heartbeater.cc:511] Master 127.30.185.188:37311 requested a full tablet report, sending...
I20260811 18:41:15.335664 12728 heartbeater.cc:347] Connected to a master server at 127.30.185.190:39547
I20260811 18:41:15.335829 12728 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:15.336226 12742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:15.290623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:41891 (local address 127.30.185.130:52199)
0811 18:41:15.297872 (+ 7249us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:15.297897 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:15.302668 (+ 4771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:15.303820 (+ 1152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:15.303836 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:15.305460 (+ 1624us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:15.307019 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:15.307035 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.308820 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.308825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:15.311631 (+ 2806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:15.311646 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.313023 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.313033 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:15.313410 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:15.314341 (+ 931us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:15.314386 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:15.324623 (+ 10237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:15.329122 (+ 4499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:15.329130 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:15.329137 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:15.329929 (+ 792us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:15.330581 (+ 652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:15.330589 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:15.330595 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:15.330927 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:15.332283 (+ 1356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:15.332287 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:15.332542 (+ 255us) client_negotiation.cc:769] Sending connection context
0811 18:41:15.335311 (+ 2769us) client_negotiation.cc:240] Negotiation successful
0811 18:41:15.335658 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7081,"mutex_wait_us":393,"spinlock_wait_cycles":18560,"thread_start_us":6852,"threads_started":1}
I20260811 18:41:15.336758 12728 heartbeater.cc:511] Master 127.30.185.190:39547 requested a full tablet report, sending...
I20260811 18:41:15.337139 12730 heartbeater.cc:347] Connected to a master server at 127.30.185.189:41891
I20260811 18:41:15.337296 12730 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:15.337945 12730 heartbeater.cc:511] Master 127.30.185.189:41891 requested a full tablet report, sending...
I20260811 18:41:15.337998 12050 ts_manager.cc:194] Registered new tserver with Master: ce2d62517cd247bd98c6e31ea0f88cc8 (127.30.185.130:44261)
I20260811 18:41:15.338117 10517 ts_manager.cc:194] Re-registered known tserver with Master: ce2d62517cd247bd98c6e31ea0f88cc8 (127.30.185.130:44261)
I20260811 18:41:15.339167 11667 ts_manager.cc:194] Registered new tserver with Master: ce2d62517cd247bd98c6e31ea0f88cc8 (127.30.185.130:44261)
I20260811 18:41:15.339658 10517 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:49723
I20260811 18:41:15.419258 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 11147
I20260811 18:41:15.442972 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:42453
--local_ip_for_outbound_sockets=127.30.185.131
--tserver_master_addrs=127.30.185.190:39547
--webserver_port=38665
--webserver_interface=127.30.185.131
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.131.keytab
--principal=oryx/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.185.190:39547,127.30.185.189:41891
--tserver_master_addrs=127.30.185.190:39547,127.30.185.189:41891,127.30.185.188:37311 with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:41:15.664713 12416 heartbeater.cc:503] Master 127.30.185.190:39547 was elected leader, sending a full tablet report...
W20260811 18:41:15.803474 12747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:15.803818 12747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:15.804021 12747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:15.812609 12747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:15.812803 12747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:15.812974 12747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:41:15.823462 12747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.131.keytab
--principal=oryx/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:42453
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=38665
--tserver_master_addrs=127.30.185.190:39547,127.30.185.189:41891,127.30.185.188:37311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:15.824795 12747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:15.826272 12747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:15.841851 12754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:15.841900 12758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:15.841943 12756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:15.842840 12747 server_base.cc:1034] running on GCE node
I20260811 18:41:15.843573 12747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:15.844492 12747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:15.845698 12747 hybrid_clock.cc:648] HybridClock initialized: now 1786473675845655 us; error 35 us; skew 500 ppm
I20260811 18:41:15.846043 12747 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:15 dist-test-slave-z8x9 krb5kdc[10417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473675, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:15.849525 12747 init.cc:377] Logged in from keytab as oryx/127.30.185.131@KRBTEST.COM (short username oryx)
I20260811 18:41:15.851050 12747 webserver.cc:460] Webserver started at http://127.30.185.131:38665/ using document root <none> and password file <none>
I20260811 18:41:15.851789 12747 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:15.851949 12747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:15.857717 12747 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:41:15.860615 12773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:15.862068 12747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:41:15.862252 12747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "659ef4fa0cb14355b6790530f4b73142"
format_stamp: "Formatted at 2026-08-11 18:41:12 on dist-test-slave-z8x9"
server_key: "5bd4b2c4d8e60ea01056fbbfbc9ae32a"
server_key_iv: "83f44d95ed6da9e50b9ee7e6a2d3542f"
server_key_version: "encryptionkey@0"
I20260811 18:41:15.863329 12747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:15.887723 12747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:15.892197 12747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:15.892558 12747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:15.893671 12747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:15.895146 12747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:15.895273 12747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:15.895413 12747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:15.895504 12747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:15.930801 12747 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:42453
I20260811 18:41:15.930927 13009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:42453 every 8 connection(s)
I20260811 18:41:15.932632 12747 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:41:15.941668 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 12747
Aug 11 18:41:15 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473675, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.189@KRBTEST.COM
Aug 11 18:41:15 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473675, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM
Aug 11 18:41:15 dist-test-slave-z8x9 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473675, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.188@KRBTEST.COM
I20260811 18:41:15.964972 13023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:15.936879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:38663 (local address 127.30.185.189:41891)
0811 18:41:15.937464 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:15.937471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:15.939307 (+ 1836us) server_negotiation.cc:407] Connection header received
0811 18:41:15.941504 (+ 2197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:15.941512 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:15.941602 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:15.941730 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:15.944827 (+ 3097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.945997 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.949139 (+ 3142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.949850 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.949949 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:15.955859 (+ 5910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:15.955869 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:15.955874 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:15.955937 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:15.959998 (+ 4061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:15.961150 (+ 1152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:15.961158 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:15.961162 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:15.961385 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:15.962016 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:15.962021 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:15.962024 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:15.962584 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:15.962763 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:15.963977 (+ 1214us) server_negotiation.cc:299] Negotiation successful
0811 18:41:15.964391 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":352,"threads_started":1}
I20260811 18:41:15.966207 13019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:15.936882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:41891 (local address 127.30.185.131:38663)
0811 18:41:15.939214 (+ 2332us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:15.939232 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:15.941064 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:15.942159 (+ 1095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:15.942171 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:15.943407 (+ 1236us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:15.944591 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:15.944636 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.946201 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.946207 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:15.948755 (+ 2548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:15.948771 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.950330 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.950340 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:15.950671 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:15.951756 (+ 1085us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:15.951798 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:15.955480 (+ 3682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:15.960195 (+ 4715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:15.960207 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:15.960228 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:15.960992 (+ 764us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:15.961569 (+ 577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:15.961575 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:15.961582 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:15.961862 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:15.962883 (+ 1021us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:15.962893 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:15.963400 (+ 507us) client_negotiation.cc:769] Sending connection context
0811 18:41:15.963891 (+ 491us) client_negotiation.cc:240] Negotiation successful
0811 18:41:15.964540 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2174,"mutex_wait_us":303,"spinlock_wait_cycles":70912,"thread_start_us":747,"threads_started":1}
I20260811 18:41:15.968089 13015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:15.935473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:43921 (local address 127.30.185.188:37311)
0811 18:41:15.936028 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:15.936034 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:15.938205 (+ 2171us) server_negotiation.cc:407] Connection header received
0811 18:41:15.941379 (+ 3174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:15.941385 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:15.941476 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:15.941585 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:15.944827 (+ 3242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.945997 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.949605 (+ 3608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.950282 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.950379 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:15.957843 (+ 7464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:15.957854 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:15.957860 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:15.957909 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:15.962051 (+ 4142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:15.963003 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:15.963013 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:15.963019 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:15.963255 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:15.964374 (+ 1119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:15.964379 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:15.964382 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:15.964814 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:15.965007 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:15.967260 (+ 2253us) server_negotiation.cc:299] Negotiation successful
0811 18:41:15.967541 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":246,"threads_started":1}
I20260811 18:41:15.968042 13014 heartbeater.cc:347] Connected to a master server at 127.30.185.189:41891
I20260811 18:41:15.968458 13014 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:15.968677 13024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:15.936173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37311 (local address 127.30.185.131:43921)
0811 18:41:15.938014 (+ 1841us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:15.938091 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:15.941034 (+ 2943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:15.942202 (+ 1168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:15.942206 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:15.943407 (+ 1201us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:15.944591 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:15.944636 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.946201 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.946207 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:15.949440 (+ 3233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:15.949455 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.950664 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.950670 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:15.950842 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:15.951851 (+ 1009us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:15.951876 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:15.957088 (+ 5212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:15.962217 (+ 5129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:15.962222 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:15.962226 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:15.962844 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:15.963447 (+ 603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:15.963456 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:15.963462 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:15.964220 (+ 758us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:15.965525 (+ 1305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:15.965533 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:15.966864 (+ 1331us) client_negotiation.cc:769] Sending connection context
0811 18:41:15.967071 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 18:41:15.967612 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1566,"mutex_wait_us":340,"spinlock_wait_cycles":1792,"thread_start_us":2526,"threads_started":1}
I20260811 18:41:15.969568 13014 heartbeater.cc:511] Master 127.30.185.189:41891 requested a full tablet report, sending...
I20260811 18:41:15.969728 13012 heartbeater.cc:347] Connected to a master server at 127.30.185.188:37311
I20260811 18:41:15.969926 13012 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:15.970517 13012 heartbeater.cc:511] Master 127.30.185.188:37311 requested a full tablet report, sending...
I20260811 18:41:15.971756 11667 ts_manager.cc:194] Registered new tserver with Master: 659ef4fa0cb14355b6790530f4b73142 (127.30.185.131:42453)
I20260811 18:41:15.971891 12050 ts_manager.cc:194] Registered new tserver with Master: 659ef4fa0cb14355b6790530f4b73142 (127.30.185.131:42453)
I20260811 18:41:15.973862 13025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:15.939309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.30.185.131:57709)
0811 18:41:15.939849 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:15.939863 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:15.941492 (+ 1629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:15.942363 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:15.942369 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:15.943416 (+ 1047us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:15.944599 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:15.944638 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.946576 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.946583 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:15.948953 (+ 2370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:15.948964 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.951185 (+ 2221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.951189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:15.951315 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:15.952063 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:15.952080 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:15.956021 (+ 3941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:15.962968 (+ 6947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:15.962972 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:15.962975 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:15.963579 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:15.965359 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:15.965367 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:15.965372 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:15.965687 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:15.970921 (+ 5234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:15.970926 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:15.972834 (+ 1908us) client_negotiation.cc:769] Sending connection context
0811 18:41:15.973040 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 18:41:15.973438 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"mutex_wait_us":392,"spinlock_wait_cycles":15744,"thread_start_us":313,"threads_started":1}
I20260811 18:41:15.974442 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:15.939431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:57709 (local address 127.30.185.190:39547)
0811 18:41:15.941959 (+ 2528us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:15.941966 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:15.941987 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:41:15.942052 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:15.942058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:15.942144 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:15.942672 (+ 528us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:15.944878 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.946339 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.949101 (+ 2762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.950039 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.950226 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:15.956304 (+ 6078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:15.956314 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:15.956322 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:15.956380 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:15.961944 (+ 5564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:15.963712 (+ 1768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:15.963721 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:15.963727 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:15.963936 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:15.965825 (+ 1889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:15.965834 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:15.965840 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:15.966546 (+ 706us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:15.966816 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:15.973239 (+ 6423us) server_negotiation.cc:299] Negotiation successful
0811 18:41:15.973580 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2388,"thread_start_us":2292,"threads_started":1}
I20260811 18:41:15.975286 13013 heartbeater.cc:347] Connected to a master server at 127.30.185.190:39547
I20260811 18:41:15.975427 13013 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:15.976432 13013 heartbeater.cc:511] Master 127.30.185.190:39547 requested a full tablet report, sending...
I20260811 18:41:15.977645 10517 ts_manager.cc:194] Re-registered known tserver with Master: 659ef4fa0cb14355b6790530f4b73142 (127.30.185.131:42453)
I20260811 18:41:15.978922 10517 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:57709
I20260811 18:41:15.999424 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:15.973225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42402 (local address 127.30.185.190:39547)
0811 18:41:15.975057 (+ 1832us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:15.975063 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:15.976793 (+ 1730us) server_negotiation.cc:407] Connection header received
0811 18:41:15.977140 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:15.977146 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:15.977303 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:15.977416 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:15.984311 (+ 6895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.985614 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.988402 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.988927 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.989071 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:15.993197 (+ 4126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:15.993208 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:15.993213 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:15.993258 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:15.996206 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:15.997094 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:15.997100 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:15.997104 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:15.997326 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:15.997725 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:15.997731 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:15.997735 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:15.998164 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:15.998419 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:15.998808 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 18:41:15.999054 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1731,"thread_start_us":7865,"threads_started":1}
I20260811 18:41:16.003708 13015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:15.979904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53264 (local address 127.30.185.188:37311)
0811 18:41:15.980160 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:15.980169 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:15.984285 (+ 4116us) server_negotiation.cc:407] Connection header received
0811 18:41:15.984535 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:15.984541 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:15.984620 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:15.984727 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:15.987479 (+ 2752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.988460 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.990547 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.991419 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.991572 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:15.995865 (+ 4293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:15.995874 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:15.995880 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:15.995929 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:16.000097 (+ 4168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:16.000956 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:16.000964 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:16.000971 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:16.001227 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:16.001716 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:16.001722 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:16.001727 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:16.002082 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:16.002442 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:16.003005 (+ 563us) server_negotiation.cc:299] Negotiation successful
0811 18:41:16.003289 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 18:41:16.005055 13023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:15.978063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43838 (local address 127.30.185.189:41891)
0811 18:41:15.980616 (+ 2553us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:15.980621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:15.984824 (+ 4203us) server_negotiation.cc:407] Connection header received
0811 18:41:15.985138 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:15.985145 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:15.985258 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:15.985330 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:15.988453 (+ 3123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.989714 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.991607 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:15.992419 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:15.992537 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:15.996696 (+ 4159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:15.996703 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:15.996707 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:15.996754 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:16.001161 (+ 4407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:16.002093 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:16.002099 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:16.002102 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:16.002334 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:16.002942 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:16.002947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:16.002949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:16.003537 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:16.003767 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:16.004432 (+ 665us) server_negotiation.cc:299] Negotiation successful
0811 18:41:16.004694 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2398}
I20260811 18:41:16.004601 10517 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:41:16.013069 10517 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:41:16.050686 13067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:16.037213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:44261 (local address 127.0.0.1:35726)
0811 18:41:16.038540 (+ 1327us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:16.038557 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:16.038677 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:16.040415 (+ 1738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:16.040420 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:16.040447 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:16.040950 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:16.040966 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.043892 (+ 2926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.043896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:16.046757 (+ 2861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:16.046783 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.049125 (+ 2342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.049131 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:16.050026 (+ 895us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:16.050060 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 18:41:16.050138 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 18:41:16.050275 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1127,"mutex_wait_us":3,"thread_start_us":4866,"threads_started":1}
I20260811 18:41:16.051046 13066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:16.037486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35726 (local address 127.30.185.130:44261)
0811 18:41:16.039802 (+ 2316us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:16.039808 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:16.039831 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:41:16.039913 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:16.039919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:16.040120 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:16.040437 (+ 317us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:16.041792 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.043591 (+ 1799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.046950 (+ 3359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.048916 (+ 1966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.049080 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:16.049348 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:16.050241 (+ 893us) server_negotiation.cc:299] Negotiation successful
0811 18:41:16.050420 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2166,"thread_start_us":2053,"threads_started":1}
I20260811 18:41:16.052505 13061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:16.036327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:34579 (local address 127.0.0.1:33188)
0811 18:41:16.037378 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:16.037436 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:16.037604 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:16.043592 (+ 5988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:16.043599 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:16.043629 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:16.044610 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:16.044626 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.046980 (+ 2354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.046986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:16.049765 (+ 2779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:16.049780 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.051623 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.051629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:16.051817 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:16.051862 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 18:41:16.051964 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 18:41:16.052102 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"mutex_wait_us":73,"thread_start_us":423,"threads_started":1}
I20260811 18:41:16.052520 13062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:16.036558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33188 (local address 127.30.185.129:34579)
0811 18:41:16.041572 (+ 5014us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:16.041583 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:16.041615 (+ 32us) server_negotiation.cc:407] Connection header received
0811 18:41:16.042992 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:16.043005 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:16.043331 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:16.043623 (+ 292us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:16.045064 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.046835 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.049926 (+ 3091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.051420 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.051612 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:16.051771 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:16.052046 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 18:41:16.052211 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4784,"thread_start_us":4643,"threads_started":1}
I20260811 18:41:16.052807 13065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:16.037610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:42453 (local address 127.0.0.1:51590)
0811 18:41:16.042527 (+ 4917us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:16.042546 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:16.042749 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:16.043996 (+ 1247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:16.044003 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:16.044035 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:16.044802 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:16.044815 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.047025 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.047029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:16.048979 (+ 1950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:16.049015 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.051934 (+ 2919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.051943 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:16.052110 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:16.052137 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 18:41:16.052239 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 18:41:16.052372 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4731,"mutex_wait_us":2,"thread_start_us":1219,"threads_started":1}
I20260811 18:41:16.055476 12582 tablet_service.cc:1467] Processing CreateTablet for tablet 31236cc5f3224781b0c50f00b00e3968 (DEFAULT_TABLE table=test-table [id=4329177f8d754725a1bfff8be05721f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:41:16.056231 12273 tablet_service.cc:1467] Processing CreateTablet for tablet 31236cc5f3224781b0c50f00b00e3968 (DEFAULT_TABLE table=test-table [id=4329177f8d754725a1bfff8be05721f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:41:16.056691 12582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31236cc5f3224781b0c50f00b00e3968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:16.057444 12273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31236cc5f3224781b0c50f00b00e3968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:16.058252 13068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:16.038284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51590 (local address 127.30.185.131:42453)
0811 18:41:16.039252 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:16.039263 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:16.042677 (+ 3414us) server_negotiation.cc:407] Connection header received
0811 18:41:16.042936 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:16.042956 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:16.043296 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:16.043472 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:16.045035 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.046619 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.049309 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.050729 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.050943 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:16.051129 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:16.057714 (+ 6585us) server_negotiation.cc:299] Negotiation successful
0811 18:41:16.057902 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":783,"thread_start_us":963,"threads_started":1}
I20260811 18:41:16.062002 12877 tablet_service.cc:1467] Processing CreateTablet for tablet 31236cc5f3224781b0c50f00b00e3968 (DEFAULT_TABLE table=test-table [id=4329177f8d754725a1bfff8be05721f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:41:16.063728 12877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31236cc5f3224781b0c50f00b00e3968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:16.077814 13072 tablet_bootstrap.cc:492] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8: Bootstrap starting.
I20260811 18:41:16.080685 13074 tablet_bootstrap.cc:492] T 31236cc5f3224781b0c50f00b00e3968 P b92d06495a8d4bd1a82526fe9071c9d1: Bootstrap starting.
I20260811 18:41:16.083832 13072 tablet_bootstrap.cc:654] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:16.085457 13074 tablet_bootstrap.cc:654] T 31236cc5f3224781b0c50f00b00e3968 P b92d06495a8d4bd1a82526fe9071c9d1: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:16.085445 13072 log.cc:824] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8: Log is configured to *not* fsync() on all Append() calls
I20260811 18:41:16.087745 13074 log.cc:824] T 31236cc5f3224781b0c50f00b00e3968 P b92d06495a8d4bd1a82526fe9071c9d1: Log is configured to *not* fsync() on all Append() calls
I20260811 18:41:16.090232 13075 tablet_bootstrap.cc:492] T 31236cc5f3224781b0c50f00b00e3968 P 659ef4fa0cb14355b6790530f4b73142: Bootstrap starting.
I20260811 18:41:16.094296 13074 tablet_bootstrap.cc:492] T 31236cc5f3224781b0c50f00b00e3968 P b92d06495a8d4bd1a82526fe9071c9d1: No bootstrap required, opened a new log
I20260811 18:41:16.094615 13074 ts_tablet_manager.cc:1397] T 31236cc5f3224781b0c50f00b00e3968 P b92d06495a8d4bd1a82526fe9071c9d1: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.009s
I20260811 18:41:16.095975 13075 tablet_bootstrap.cc:654] T 31236cc5f3224781b0c50f00b00e3968 P 659ef4fa0cb14355b6790530f4b73142: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:16.097712 13075 log.cc:824] T 31236cc5f3224781b0c50f00b00e3968 P 659ef4fa0cb14355b6790530f4b73142: Log is configured to *not* fsync() on all Append() calls
I20260811 18:41:16.099876 13072 tablet_bootstrap.cc:492] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8: No bootstrap required, opened a new log
I20260811 18:41:16.100510 13072 ts_tablet_manager.cc:1397] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.013s
I20260811 18:41:16.101683 13075 tablet_bootstrap.cc:492] T 31236cc5f3224781b0c50f00b00e3968 P 659ef4fa0cb14355b6790530f4b73142: No bootstrap required, opened a new log
I20260811 18:41:16.102003 13075 ts_tablet_manager.cc:1397] T 31236cc5f3224781b0c50f00b00e3968 P 659ef4fa0cb14355b6790530f4b73142: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.004s
I20260811 18:41:16.108251 13074 raft_consensus.cc:348] T 31236cc5f3224781b0c50f00b00e3968 P b92d06495a8d4bd1a82526fe9071c9d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34579 } } peers { permanent_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44261 } } peers { permanent_uuid: "659ef4fa0cb14355b6790530f4b73142" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42453 } }
I20260811 18:41:16.108758 13074 raft_consensus.cc:374] T 31236cc5f3224781b0c50f00b00e3968 P b92d06495a8d4bd1a82526fe9071c9d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:16.108866 13074 raft_consensus.cc:729] T 31236cc5f3224781b0c50f00b00e3968 P b92d06495a8d4bd1a82526fe9071c9d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b92d06495a8d4bd1a82526fe9071c9d1, State: Initialized, Role: FOLLOWER
I20260811 18:41:16.109319 13072 raft_consensus.cc:348] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34579 } } peers { permanent_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44261 } } peers { permanent_uuid: "659ef4fa0cb14355b6790530f4b73142" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42453 } }
I20260811 18:41:16.109551 13074 consensus_queue.cc:260] T 31236cc5f3224781b0c50f00b00e3968 P b92d06495a8d4bd1a82526fe9071c9d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34579 } } peers { permanent_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44261 } } peers { permanent_uuid: "659ef4fa0cb14355b6790530f4b73142" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42453 } }
I20260811 18:41:16.109794 13072 raft_consensus.cc:374] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:16.109899 13072 raft_consensus.cc:729] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce2d62517cd247bd98c6e31ea0f88cc8, State: Initialized, Role: FOLLOWER
I20260811 18:41:16.110476 13072 consensus_queue.cc:260] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34579 } } peers { permanent_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44261 } } peers { permanent_uuid: "659ef4fa0cb14355b6790530f4b73142" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42453 } }
I20260811 18:41:16.113049 12728 heartbeater.cc:503] Master 127.30.185.190:39547 was elected leader, sending a full tablet report...
I20260811 18:41:16.118166 13075 raft_consensus.cc:348] T 31236cc5f3224781b0c50f00b00e3968 P 659ef4fa0cb14355b6790530f4b73142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34579 } } peers { permanent_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44261 } } peers { permanent_uuid: "659ef4fa0cb14355b6790530f4b73142" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42453 } }
I20260811 18:41:16.118832 13075 raft_consensus.cc:374] T 31236cc5f3224781b0c50f00b00e3968 P 659ef4fa0cb14355b6790530f4b73142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:16.118950 13075 raft_consensus.cc:729] T 31236cc5f3224781b0c50f00b00e3968 P 659ef4fa0cb14355b6790530f4b73142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 659ef4fa0cb14355b6790530f4b73142, State: Initialized, Role: FOLLOWER
I20260811 18:41:16.119884 13075 consensus_queue.cc:260] T 31236cc5f3224781b0c50f00b00e3968 P 659ef4fa0cb14355b6790530f4b73142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34579 } } peers { permanent_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44261 } } peers { permanent_uuid: "659ef4fa0cb14355b6790530f4b73142" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42453 } }
I20260811 18:41:16.120380 13072 ts_tablet_manager.cc:1428] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8: Time spent starting tablet: real 0.020s user 0.014s sys 0.000s
I20260811 18:41:16.124342 13013 heartbeater.cc:503] Master 127.30.185.190:39547 was elected leader, sending a full tablet report...
I20260811 18:41:16.127179 13074 ts_tablet_manager.cc:1428] T 31236cc5f3224781b0c50f00b00e3968 P b92d06495a8d4bd1a82526fe9071c9d1: Time spent starting tablet: real 0.032s user 0.016s sys 0.015s
I20260811 18:41:16.129179 13075 ts_tablet_manager.cc:1428] T 31236cc5f3224781b0c50f00b00e3968 P 659ef4fa0cb14355b6790530f4b73142: Time spent starting tablet: real 0.027s user 0.013s sys 0.004s
I20260811 18:41:16.150600 13087 raft_consensus.cc:482] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:41:16.150964 13087 raft_consensus.cc:504] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34579 } } peers { permanent_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44261 } } peers { permanent_uuid: "659ef4fa0cb14355b6790530f4b73142" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42453 } }
I20260811 18:41:16.155155 13087 leader_election.cc:288] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b92d06495a8d4bd1a82526fe9071c9d1 (127.30.185.129:34579), 659ef4fa0cb14355b6790530f4b73142 (127.30.185.131:42453)
I20260811 18:41:16.163241 13068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:16.153739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:44513 (local address 127.30.185.131:42453)
0811 18:41:16.154148 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:16.154153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:16.155211 (+ 1058us) server_negotiation.cc:407] Connection header received
0811 18:41:16.155594 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:16.155613 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:16.155735 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:16.155860 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:16.157255 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.158481 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.161213 (+ 2732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.162043 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.162179 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:16.162235 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:16.162800 (+ 565us) server_negotiation.cc:299] Negotiation successful
0811 18:41:16.162957 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288}
I20260811 18:41:16.163329 13091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:16.153678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:42453 (local address 127.30.185.130:44513)
0811 18:41:16.155074 (+ 1396us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:16.155097 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:16.155359 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:16.156141 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:16.156146 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:16.156191 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:16.157036 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:16.157054 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.158664 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.158669 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:16.160922 (+ 2253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:16.160940 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.162327 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.162334 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:16.162554 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:16.162592 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 18:41:16.162725 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 18:41:16.162886 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1243,"mutex_wait_us":1,"thread_start_us":2125,"threads_started":1}
I20260811 18:41:16.163981 13089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:16.153921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:34579 (local address 127.30.185.130:36155)
0811 18:41:16.156391 (+ 2470us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:16.156408 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:16.156525 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:16.156984 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:16.156987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:16.157009 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:16.157586 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:16.157600 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.158848 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.158852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:16.161033 (+ 2181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:16.161054 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.162901 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.162909 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:16.163105 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:16.163198 (+ 93us) client_negotiation.cc:769] Sending connection context
0811 18:41:16.163324 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 18:41:16.163472 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2281,"mutex_wait_us":22,"thread_start_us":687,"threads_started":1}
I20260811 18:41:16.164235 13062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:16.153739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:36155 (local address 127.30.185.129:34579)
0811 18:41:16.154000 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:16.154005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:16.156573 (+ 2568us) server_negotiation.cc:407] Connection header received
0811 18:41:16.156704 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:16.156710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:16.156845 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:16.156979 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:16.157860 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.158631 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.161292 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.162616 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.162781 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:16.162849 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:16.163807 (+ 958us) server_negotiation.cc:299] Negotiation successful
0811 18:41:16.163958 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260811 18:41:16.164546 12913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31236cc5f3224781b0c50f00b00e3968" candidate_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "659ef4fa0cb14355b6790530f4b73142" is_pre_election: true
I20260811 18:41:16.165232 12913 raft_consensus.cc:2454] T 31236cc5f3224781b0c50f00b00e3968 P 659ef4fa0cb14355b6790530f4b73142 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce2d62517cd247bd98c6e31ea0f88cc8 in term 0.
I20260811 18:41:16.165309 12315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31236cc5f3224781b0c50f00b00e3968" candidate_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" is_pre_election: true
I20260811 18:41:16.165899 12315 raft_consensus.cc:2454] T 31236cc5f3224781b0c50f00b00e3968 P b92d06495a8d4bd1a82526fe9071c9d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce2d62517cd247bd98c6e31ea0f88cc8 in term 0.
I20260811 18:41:16.166162 12497 leader_election.cc:302] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 659ef4fa0cb14355b6790530f4b73142, ce2d62517cd247bd98c6e31ea0f88cc8; no voters:
I20260811 18:41:16.166906 13087 raft_consensus.cc:2781] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:41:16.167104 13087 raft_consensus.cc:482] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:41:16.167245 13087 raft_consensus.cc:3037] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:16.169821 13087 raft_consensus.cc:504] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34579 } } peers { permanent_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44261 } } peers { permanent_uuid: "659ef4fa0cb14355b6790530f4b73142" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42453 } }
I20260811 18:41:16.170476 13087 leader_election.cc:288] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8 [CANDIDATE]: Term 1 election: Requested vote from peers b92d06495a8d4bd1a82526fe9071c9d1 (127.30.185.129:34579), 659ef4fa0cb14355b6790530f4b73142 (127.30.185.131:42453)
I20260811 18:41:16.171008 12315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31236cc5f3224781b0c50f00b00e3968" candidate_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b92d06495a8d4bd1a82526fe9071c9d1"
I20260811 18:41:16.171135 12913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31236cc5f3224781b0c50f00b00e3968" candidate_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "659ef4fa0cb14355b6790530f4b73142"
I20260811 18:41:16.171247 12315 raft_consensus.cc:3037] T 31236cc5f3224781b0c50f00b00e3968 P b92d06495a8d4bd1a82526fe9071c9d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:16.171483 12913 raft_consensus.cc:3037] T 31236cc5f3224781b0c50f00b00e3968 P 659ef4fa0cb14355b6790530f4b73142 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:16.173877 12315 raft_consensus.cc:2454] T 31236cc5f3224781b0c50f00b00e3968 P b92d06495a8d4bd1a82526fe9071c9d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce2d62517cd247bd98c6e31ea0f88cc8 in term 1.
I20260811 18:41:16.174126 12913 raft_consensus.cc:2454] T 31236cc5f3224781b0c50f00b00e3968 P 659ef4fa0cb14355b6790530f4b73142 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce2d62517cd247bd98c6e31ea0f88cc8 in term 1.
I20260811 18:41:16.174724 12492 leader_election.cc:302] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b92d06495a8d4bd1a82526fe9071c9d1, ce2d62517cd247bd98c6e31ea0f88cc8; no voters:
I20260811 18:41:16.175123 13087 raft_consensus.cc:2781] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:41:16.175750 13087 raft_consensus.cc:686] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 1 LEADER]: Becoming Leader. State: Replica: ce2d62517cd247bd98c6e31ea0f88cc8, State: Running, Role: LEADER
I20260811 18:41:16.176327 13087 consensus_queue.cc:237] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34579 } } peers { permanent_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44261 } } peers { permanent_uuid: "659ef4fa0cb14355b6790530f4b73142" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42453 } }
I20260811 18:41:16.182497 10517 catalog_manager.cc:5495] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce2d62517cd247bd98c6e31ea0f88cc8 (127.30.185.130). New cstate: current_term: 1 leader_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "659ef4fa0cb14355b6790530f4b73142" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42453 } health_report { overall_health: UNKNOWN } } }
W20260811 18:41:16.187230 13018 tablet.cc:2369] T 31236cc5f3224781b0c50f00b00e3968 P 659ef4fa0cb14355b6790530f4b73142: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:41:16.224812 13066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:16.218735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35740 (local address 127.30.185.130:44261)
0811 18:41:16.218962 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:16.218971 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:16.219136 (+ 165us) server_negotiation.cc:407] Connection header received
0811 18:41:16.219364 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:16.219368 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:16.219459 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:16.219591 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:41:16.220120 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.220908 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.222576 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.223000 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.223096 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:16.223590 (+ 494us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:41:16.224138 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:41:16.224246 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:16.224382 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 18:41:16.224450 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:41:16.234666 12315 raft_consensus.cc:1261] T 31236cc5f3224781b0c50f00b00e3968 P b92d06495a8d4bd1a82526fe9071c9d1 [term 1 FOLLOWER]: Refusing update from remote peer ce2d62517cd247bd98c6e31ea0f88cc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:41:16.234799 12913 raft_consensus.cc:1261] T 31236cc5f3224781b0c50f00b00e3968 P 659ef4fa0cb14355b6790530f4b73142 [term 1 FOLLOWER]: Refusing update from remote peer ce2d62517cd247bd98c6e31ea0f88cc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:41:16.235837 13092 consensus_queue.cc:1035] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:41:16.236354 13087 consensus_queue.cc:1035] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "659ef4fa0cb14355b6790530f4b73142" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:41:16.264568 13104 mvcc.cc:205] Tried to move back new op lower bound from 7317396177845583872 to 7317396177622585344. Current Snapshot: MvccSnapshot[applied={T|T < 7317396177845583872}]
I20260811 18:41:16.293422 10510 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:41:16.293799 10510 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:41:16.300949 10510 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0691bb2c526b42a697988a418b6d8750: Sending DeleteTablet for 3 replicas of tablet 31236cc5f3224781b0c50f00b00e3968
I20260811 18:41:16.302310 12273 tablet_service.cc:1514] Processing DeleteTablet for tablet 31236cc5f3224781b0c50f00b00e3968 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:41:16 UTC) from {username='oryx', principal='oryx/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:33188
I20260811 18:41:16.302711 12582 tablet_service.cc:1514] Processing DeleteTablet for tablet 31236cc5f3224781b0c50f00b00e3968 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:41:16 UTC) from {username='oryx', principal='oryx/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:35726
I20260811 18:41:16.303716 12877 tablet_service.cc:1514] Processing DeleteTablet for tablet 31236cc5f3224781b0c50f00b00e3968 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:41:16 UTC) from {username='oryx', principal='oryx/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:51590
I20260811 18:41:16.303985 13119 tablet_replica.cc:330] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8: stopping tablet replica
I20260811 18:41:16.304455 13119 raft_consensus.cc:2229] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:41:16.305222 13119 raft_consensus.cc:2258] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:41:16.305605 13117 tablet_replica.cc:330] T 31236cc5f3224781b0c50f00b00e3968 P b92d06495a8d4bd1a82526fe9071c9d1: stopping tablet replica
I20260811 18:41:16.306419 13117 raft_consensus.cc:2229] T 31236cc5f3224781b0c50f00b00e3968 P b92d06495a8d4bd1a82526fe9071c9d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:41:16.306968 13117 raft_consensus.cc:2258] T 31236cc5f3224781b0c50f00b00e3968 P b92d06495a8d4bd1a82526fe9071c9d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:41:16.307788 13119 ts_tablet_manager.cc:1905] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:41:16.308243 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 10444
I20260811 18:41:16.308634 13121 tablet_replica.cc:330] T 31236cc5f3224781b0c50f00b00e3968 P 659ef4fa0cb14355b6790530f4b73142: stopping tablet replica
I20260811 18:41:16.309537 10583 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:16.310977 13117 ts_tablet_manager.cc:1905] T 31236cc5f3224781b0c50f00b00e3968 P b92d06495a8d4bd1a82526fe9071c9d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:41:16.316851 13121 raft_consensus.cc:2229] T 31236cc5f3224781b0c50f00b00e3968 P 659ef4fa0cb14355b6790530f4b73142 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:41:16.317636 13119 ts_tablet_manager.cc:1918] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:41:16.317863 13119 log.cc:1196] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/31236cc5f3224781b0c50f00b00e3968
I20260811 18:41:16.318604 13119 ts_tablet_manager.cc:1939] T 31236cc5f3224781b0c50f00b00e3968 P ce2d62517cd247bd98c6e31ea0f88cc8: Deleting consensus metadata
I20260811 18:41:16.320760 13121 raft_consensus.cc:2258] T 31236cc5f3224781b0c50f00b00e3968 P 659ef4fa0cb14355b6790530f4b73142 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:41:16.322435 13117 ts_tablet_manager.cc:1918] T 31236cc5f3224781b0c50f00b00e3968 P b92d06495a8d4bd1a82526fe9071c9d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:41:16.322633 13117 log.cc:1196] T 31236cc5f3224781b0c50f00b00e3968 P b92d06495a8d4bd1a82526fe9071c9d1: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/wal/wals/31236cc5f3224781b0c50f00b00e3968
I20260811 18:41:16.323526 13117 ts_tablet_manager.cc:1939] T 31236cc5f3224781b0c50f00b00e3968 P b92d06495a8d4bd1a82526fe9071c9d1: Deleting consensus metadata
I20260811 18:41:16.331270 13121 ts_tablet_manager.cc:1905] T 31236cc5f3224781b0c50f00b00e3968 P 659ef4fa0cb14355b6790530f4b73142: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:41:16.341539 13023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:16.305995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43846 (local address 127.30.185.189:41891)
0811 18:41:16.306184 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:16.306190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:16.308781 (+ 2591us) server_negotiation.cc:407] Connection header received
0811 18:41:16.308857 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:16.308864 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:16.308949 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:16.309060 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:16.320026 (+ 10966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.321537 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.324549 (+ 3012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.325092 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.325272 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:16.332908 (+ 7636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:16.332914 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:16.332917 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:16.332951 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:16.337665 (+ 4714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:16.338757 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:16.338764 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:16.338767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:16.338956 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:16.339585 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:16.339591 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:16.339594 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:16.340013 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:16.340256 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:16.340907 (+ 651us) server_negotiation.cc:299] Negotiation successful
0811 18:41:16.341152 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 18:41:16.341988 13121 ts_tablet_manager.cc:1918] T 31236cc5f3224781b0c50f00b00e3968 P 659ef4fa0cb14355b6790530f4b73142: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:41:16.342211 13121 log.cc:1196] T 31236cc5f3224781b0c50f00b00e3968 P 659ef4fa0cb14355b6790530f4b73142: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/31236cc5f3224781b0c50f00b00e3968
I20260811 18:41:16.343058 13121 ts_tablet_manager.cc:1939] T 31236cc5f3224781b0c50f00b00e3968 P 659ef4fa0cb14355b6790530f4b73142: Deleting consensus metadata
I20260811 18:41:16.360471 13015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:16.307391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53268 (local address 127.30.185.188:37311)
0811 18:41:16.307570 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:16.307577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:16.344984 (+ 37407us) server_negotiation.cc:407] Connection header received
0811 18:41:16.345311 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:16.345317 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:16.345404 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:16.345460 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:16.347180 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.348093 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.349845 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.350265 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.350410 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:16.353967 (+ 3557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:16.353973 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:16.353976 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:16.354017 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:16.356985 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:16.357913 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:16.357917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:16.357919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:16.358062 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:16.358605 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:16.358609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:16.358611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:16.358953 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:16.359306 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:16.359605 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 18:41:16.360127 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:41:16.526717 10487 catalog_manager.cc:4841] TS 659ef4fa0cb14355b6790530f4b73142 (127.30.185.131:42453): tablet 31236cc5f3224781b0c50f00b00e3968 (table test-table [id=4329177f8d754725a1bfff8be05721f7]) successfully deleted
I20260811 18:41:16.526840 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 10444
W20260811 18:41:16.558248 12497 connection.cc:535] server connection from 127.0.0.1:35726 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:41:16.583154 13015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:16.564349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53276 (local address 127.30.185.188:37311)
0811 18:41:16.564683 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:16.564690 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:16.564999 (+ 309us) server_negotiation.cc:407] Connection header received
0811 18:41:16.565332 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:16.565339 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:16.565425 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:16.565502 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:16.569534 (+ 4032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.570673 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.572536 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.573062 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.573261 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:16.576584 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:16.576595 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:16.576601 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:16.576659 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:16.580136 (+ 3477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:16.580897 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:16.580903 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:16.580909 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:16.581049 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:16.581471 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:16.581476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:16.581479 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:16.581810 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:16.581986 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:16.582536 (+ 550us) server_negotiation.cc:299] Negotiation successful
0811 18:41:16.582781 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 18:41:16.583770 13023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:16.562977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43854 (local address 127.30.185.189:41891)
0811 18:41:16.563300 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:16.563310 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:16.564336 (+ 1026us) server_negotiation.cc:407] Connection header received
0811 18:41:16.565380 (+ 1044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:16.565386 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:16.565462 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:16.565532 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:16.568116 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.569381 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.571128 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:16.571624 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:16.571771 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:16.575547 (+ 3776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:16.575552 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:16.575556 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:16.575591 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:16.579424 (+ 3833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:16.580238 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:16.580244 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:16.580247 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:16.580400 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:16.580829 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:16.580835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:16.580838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:16.581304 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:16.581569 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:16.583204 (+ 1635us) server_negotiation.cc:299] Negotiation successful
0811 18:41:16.583452 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 18:41:17.264024 13148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.263310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.30.185.129:42439)
0811 18:41:17.263722 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:17.263845 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":215,"threads_started":1}
W20260811 18:41:17.264501 12416 heartbeater.cc:644] Failed to heartbeat to 127.30.185.190:39547 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
I20260811 18:41:17.266445 13151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.265663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.30.185.130:40399)
0811 18:41:17.266087 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:17.266208 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":212,"threads_started":1}
W20260811 18:41:17.267001 12728 heartbeater.cc:644] Failed to heartbeat to 127.30.185.190:39547 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
I20260811 18:41:17.284508 13152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.282736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.30.185.131:43717)
0811 18:41:17.284094 (+ 1358us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:17.284240 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1208,"thread_start_us":1092,"threads_started":1}
W20260811 18:41:17.285007 13013 heartbeater.cc:644] Failed to heartbeat to 127.30.185.190:39547 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
I20260811 18:41:17.802556 13154 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0691bb2c526b42a697988a418b6d8750)
I20260811 18:41:17.802688 13156 raft_consensus.cc:482] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0691bb2c526b42a697988a418b6d8750)
I20260811 18:41:17.802832 13154 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: false } }
I20260811 18:41:17.802939 13156 raft_consensus.cc:504] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: false } }
I20260811 18:41:17.803819 13154 leader_election.cc:288] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0691bb2c526b42a697988a418b6d8750 (127.30.185.190:39547), eee2ef3d5cd1440eafaa7cd2f3a4fccc (127.30.185.188:37311)
I20260811 18:41:17.803819 13156 leader_election.cc:288] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0691bb2c526b42a697988a418b6d8750 (127.30.185.190:39547), ef1b11a131674161bb732b9c0300aae9 (127.30.185.189:41891)
I20260811 18:41:17.818202 13159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.804088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.0.0.1:42820)
0811 18:41:17.817580 (+ 13492us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:17.817846 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13346,"threads_started":1}
I20260811 18:41:17.819662 13163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.804193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.0.0.1:42832)
0811 18:41:17.818181 (+ 13988us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:17.818401 (+ 220us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13848,"thread_start_us":13720,"threads_started":1}
W20260811 18:41:17.820555 11651 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
W20260811 18:41:17.822098 12016 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
I20260811 18:41:17.828192 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.827453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.0.0.1:42854)
0811 18:41:17.827665 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:17.827890 (+ 225us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 18:41:17.828225 13159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.804117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:41891 (local address 127.0.0.1:43870)
0811 18:41:17.818650 (+ 14533us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:17.818678 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:17.818851 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:17.819685 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:17.819692 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:17.819729 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:17.820161 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:17.820176 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:17.822050 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:17.822059 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:17.825012 (+ 2953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:17.825032 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:17.827271 (+ 2239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:17.827291 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:17.827486 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:17.827531 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 18:41:17.827629 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 18:41:17.827811 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14400,"thread_start_us":13306,"threads_started":1}
W20260811 18:41:17.828439 12016 leader_election.cc:334] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0691bb2c526b42a697988a418b6d8750 (127.30.185.190:39547): Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
I20260811 18:41:17.829149 13173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.827644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.0.0.1:42866)
0811 18:41:17.828329 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:17.828783 (+ 454us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":329,"threads_started":1}
I20260811 18:41:17.829584 13164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.804184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43870 (local address 127.30.185.189:41891)
0811 18:41:17.817386 (+ 13202us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:17.817394 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:17.818985 (+ 1591us) server_negotiation.cc:407] Connection header received
0811 18:41:17.819081 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:17.819087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:17.819194 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:17.819344 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:17.820397 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:17.821822 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:17.825292 (+ 3470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:17.826822 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:17.826981 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:17.827174 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:17.828887 (+ 1713us) server_negotiation.cc:299] Negotiation successful
0811 18:41:17.829066 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":13024,"thread_start_us":12694,"threads_started":1}
W20260811 18:41:17.829717 11651 leader_election.cc:334] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0691bb2c526b42a697988a418b6d8750 (127.30.185.190:39547): Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
I20260811 18:41:17.830199 11689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ef1b11a131674161bb732b9c0300aae9" is_pre_election: true
I20260811 18:41:17.830539 11689 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eee2ef3d5cd1440eafaa7cd2f3a4fccc in term 1.
I20260811 18:41:17.831067 12021 leader_election.cc:302] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: eee2ef3d5cd1440eafaa7cd2f3a4fccc, ef1b11a131674161bb732b9c0300aae9; no voters: 0691bb2c526b42a697988a418b6d8750
I20260811 18:41:17.831442 13156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 18:41:17.831583 13156 raft_consensus.cc:482] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0691bb2c526b42a697988a418b6d8750)
I20260811 18:41:17.831710 13156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:41:17.832844 13163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.820194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37311 (local address 127.0.0.1:53290)
0811 18:41:17.821188 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:17.821206 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:17.821323 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:17.824904 (+ 3581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:17.824913 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:17.824950 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:17.825371 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:17.825382 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:17.828249 (+ 2867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:17.828255 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:17.830557 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:17.830571 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:17.832069 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:17.832078 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:17.832273 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:17.832308 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 18:41:17.832417 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 18:41:17.832577 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861}
I20260811 18:41:17.832885 13166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.818926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53290 (local address 127.30.185.188:37311)
0811 18:41:17.823838 (+ 4912us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:17.823847 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:17.823870 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:41:17.823931 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:17.823936 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:17.824018 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:17.825574 (+ 1556us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:17.825853 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:17.827177 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:17.830752 (+ 3575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:17.831865 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:17.832029 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:17.832083 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:17.832464 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 18:41:17.832577 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4763,"thread_start_us":4618,"threads_started":1}
I20260811 18:41:17.833532 12069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ef1b11a131674161bb732b9c0300aae9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" is_pre_election: true
I20260811 18:41:17.834833 13156 raft_consensus.cc:504] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: false } }
I20260811 18:41:17.835412 12069 raft_consensus.cc:2379] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ef1b11a131674161bb732b9c0300aae9 in current term 2: Already voted for candidate eee2ef3d5cd1440eafaa7cd2f3a4fccc in this term.
I20260811 18:41:17.835529 13156 leader_election.cc:288] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [CANDIDATE]: Term 2 election: Requested vote from peers 0691bb2c526b42a697988a418b6d8750 (127.30.185.190:39547), ef1b11a131674161bb732b9c0300aae9 (127.30.185.189:41891)
I20260811 18:41:17.836202 13159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.835691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.0.0.1:42874)
0811 18:41:17.835877 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:17.835994 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 18:41:17.836160 11689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ef1b11a131674161bb732b9c0300aae9"
I20260811 18:41:17.836488 11689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:41:17.837160 11651 leader_election.cc:302] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef1b11a131674161bb732b9c0300aae9; no voters: 0691bb2c526b42a697988a418b6d8750, eee2ef3d5cd1440eafaa7cd2f3a4fccc
I20260811 18:41:17.839138 11689 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eee2ef3d5cd1440eafaa7cd2f3a4fccc in term 2.
I20260811 18:41:17.839401 13154 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:41:17.839623 12021 leader_election.cc:302] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eee2ef3d5cd1440eafaa7cd2f3a4fccc, ef1b11a131674161bb732b9c0300aae9; no voters:
I20260811 18:41:17.840047 13156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:41:17.840624 13156 raft_consensus.cc:686] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [term 2 LEADER]: Becoming Leader. State: Replica: eee2ef3d5cd1440eafaa7cd2f3a4fccc, State: Running, Role: LEADER
I20260811 18:41:17.840831 13159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.840300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.0.0.1:42888)
0811 18:41:17.840539 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:17.840685 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260811 18:41:17.841063 12016 leader_election.cc:334] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0691bb2c526b42a697988a418b6d8750 (127.30.185.190:39547): Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
I20260811 18:41:17.841110 13156 consensus_queue.cc:237] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: false } }
I20260811 18:41:17.843369 13174 sys_catalog.cc:455] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [sys.catalog]: SysCatalogTable state changed. Reason: New leader eee2ef3d5cd1440eafaa7cd2f3a4fccc. Latest consensus state: current_term: 2 leader_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: false } } }
I20260811 18:41:17.843847 13174 sys_catalog.cc:458] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:17.844516 13176 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:41:17.848579 13176 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:41:17.849498 13176 catalog_manager.cc:1259] Loaded cluster ID: 6cdce79fc4a64402b260ed180c0557fb
I20260811 18:41:17.849574 13176 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:41:17.850730 13176 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:41:17.851572 13176 catalog_manager.cc:5879] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc: Loaded TSK: 0
I20260811 18:41:17.852656 13176 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:41:17.881546 12047 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:41:17.888751 12047 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:41:17.893296 13159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.892817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.0.0.1:42908)
0811 18:41:17.893004 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:17.893082 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 18:41:17.893618 11689 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [term 2 FOLLOWER]: Refusing update from remote peer eee2ef3d5cd1440eafaa7cd2f3a4fccc: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260811 18:41:17.894198 13159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.893876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.0.0.1:42922)
0811 18:41:17.894011 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:17.894084 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 18:41:17.894369 12016 consensus_peers.cc:487] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc -> Peer 0691bb2c526b42a697988a418b6d8750 (127.30.185.190:39547): Couldn't send request to peer 0691bb2c526b42a697988a418b6d8750. Status: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:41:17.894457 13174 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 18:41:17.897447 13154 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eee2ef3d5cd1440eafaa7cd2f3a4fccc. Latest consensus state: current_term: 2 leader_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: false } } }
I20260811 18:41:17.897715 13154 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:41:17.899209 13156 sys_catalog.cc:455] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: false } } }
I20260811 18:41:17.899576 13156 sys_catalog.cc:458] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:17.901786 13154 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0691bb2c526b42a697988a418b6d8750" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39547 } } peers { permanent_uuid: "ef1b11a131674161bb732b9c0300aae9" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 41891 } attrs { promote: false } } peers { permanent_uuid: "eee2ef3d5cd1440eafaa7cd2f3a4fccc" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37311 } attrs { promote: false } } }
I20260811 18:41:17.902083 13154 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef1b11a131674161bb732b9c0300aae9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:41:17.918488 13159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.908665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:34579 (local address 127.0.0.1:33200)
0811 18:41:17.908886 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:17.908915 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:17.909071 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:17.909618 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:17.909623 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:17.909651 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:17.909861 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:17.909880 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:17.912086 (+ 2206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:17.912095 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:17.914770 (+ 2675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:17.914792 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:17.917325 (+ 2533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:17.917336 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:17.917633 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:17.917695 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 18:41:17.917783 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 18:41:17.917996 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 18:41:17.920418 13191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.908667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33200 (local address 127.30.185.129:34579)
0811 18:41:17.909221 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:17.909230 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:17.909253 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:41:17.909341 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:17.909364 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:17.909446 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:17.910064 (+ 618us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:17.910280 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:17.911876 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:17.915277 (+ 3401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:17.917013 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:17.919549 (+ 2536us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:17.919677 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:17.919746 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 18:41:17.919951 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":284,"threads_started":1}
I20260811 18:41:17.922400 12273 tablet_service.cc:1467] Processing CreateTablet for tablet 772f94c04ed94b088cc16676c32535f0 (DEFAULT_TABLE table=test-table [id=5293ff8d8bbe490e86c9836c56ae3ae9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:41:17.922996 12273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 772f94c04ed94b088cc16676c32535f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:17.929469 13159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.909450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:44261 (local address 127.0.0.1:35742)
0811 18:41:17.918809 (+ 9359us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:17.918836 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:17.918970 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:17.922555 (+ 3585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:17.922560 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:17.922608 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:17.922812 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:17.922824 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:17.924715 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:17.924724 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:17.926816 (+ 2092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:17.926832 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:17.928509 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:17.928516 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:17.928808 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:17.928844 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 18:41:17.928930 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 18:41:17.929053 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9223,"mutex_wait_us":447,"thread_start_us":11483,"threads_started":1}
I20260811 18:41:17.930080 13197 tablet_bootstrap.cc:492] T 772f94c04ed94b088cc16676c32535f0 P b92d06495a8d4bd1a82526fe9071c9d1: Bootstrap starting.
I20260811 18:41:17.931602 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.909450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:42453 (local address 127.0.0.1:51606)
0811 18:41:17.909662 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:17.909678 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:17.909787 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:17.925668 (+ 15881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:17.925673 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:17.925702 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:17.926026 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:17.926039 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:17.927358 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:17.927364 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:17.928801 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:17.928815 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:17.930777 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:17.930784 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:17.930996 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:17.931039 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 18:41:17.931106 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 18:41:17.931254 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":49}
I20260811 18:41:17.932075 13197 tablet_bootstrap.cc:654] T 772f94c04ed94b088cc16676c32535f0 P b92d06495a8d4bd1a82526fe9071c9d1: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:17.932641 13192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.909621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51606 (local address 127.30.185.131:42453)
0811 18:41:17.925129 (+ 15508us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:17.925136 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:17.925158 (+ 22us) server_negotiation.cc:407] Connection header received
0811 18:41:17.925272 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:17.925279 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:17.925481 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:17.925625 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:17.926232 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:17.927227 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:17.928979 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:17.930536 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:17.932008 (+ 1472us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:17.932091 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:17.932147 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 18:41:17.932298 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15365,"thread_start_us":16147,"threads_started":1}
I20260811 18:41:17.934582 13197 tablet_bootstrap.cc:492] T 772f94c04ed94b088cc16676c32535f0 P b92d06495a8d4bd1a82526fe9071c9d1: No bootstrap required, opened a new log
I20260811 18:41:17.934741 13197 ts_tablet_manager.cc:1397] T 772f94c04ed94b088cc16676c32535f0 P b92d06495a8d4bd1a82526fe9071c9d1: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 18:41:17.935572 13197 raft_consensus.cc:348] T 772f94c04ed94b088cc16676c32535f0 P b92d06495a8d4bd1a82526fe9071c9d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34579 } } peers { permanent_uuid: "659ef4fa0cb14355b6790530f4b73142" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42453 } } peers { permanent_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44261 } }
I20260811 18:41:17.935827 13197 raft_consensus.cc:374] T 772f94c04ed94b088cc16676c32535f0 P b92d06495a8d4bd1a82526fe9071c9d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:17.935820 12877 tablet_service.cc:1467] Processing CreateTablet for tablet 772f94c04ed94b088cc16676c32535f0 (DEFAULT_TABLE table=test-table [id=5293ff8d8bbe490e86c9836c56ae3ae9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:41:17.935936 13197 raft_consensus.cc:729] T 772f94c04ed94b088cc16676c32535f0 P b92d06495a8d4bd1a82526fe9071c9d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b92d06495a8d4bd1a82526fe9071c9d1, State: Initialized, Role: FOLLOWER
I20260811 18:41:17.936281 13193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.909713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35742 (local address 127.30.185.130:44261)
0811 18:41:17.922117 (+ 12404us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:17.922127 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:17.922155 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:41:17.922247 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:17.922256 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:17.922373 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:17.922508 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:17.922985 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:17.924496 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:17.927015 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:17.928311 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:17.928523 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:17.928602 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:17.935685 (+ 7083us) server_negotiation.cc:299] Negotiation successful
0811 18:41:17.935880 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12227,"thread_start_us":12103,"threads_started":1}
I20260811 18:41:17.936311 12877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 772f94c04ed94b088cc16676c32535f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:17.936216 13197 consensus_queue.cc:260] T 772f94c04ed94b088cc16676c32535f0 P b92d06495a8d4bd1a82526fe9071c9d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34579 } } peers { permanent_uuid: "659ef4fa0cb14355b6790530f4b73142" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42453 } } peers { permanent_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44261 } }
I20260811 18:41:17.937057 13197 ts_tablet_manager.cc:1428] T 772f94c04ed94b088cc16676c32535f0 P b92d06495a8d4bd1a82526fe9071c9d1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 18:41:17.937031 12582 tablet_service.cc:1467] Processing CreateTablet for tablet 772f94c04ed94b088cc16676c32535f0 (DEFAULT_TABLE table=test-table [id=5293ff8d8bbe490e86c9836c56ae3ae9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:41:17.937544 12582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 772f94c04ed94b088cc16676c32535f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:17.944768 13213 tablet_bootstrap.cc:492] T 772f94c04ed94b088cc16676c32535f0 P 659ef4fa0cb14355b6790530f4b73142: Bootstrap starting.
I20260811 18:41:17.947153 13213 tablet_bootstrap.cc:654] T 772f94c04ed94b088cc16676c32535f0 P 659ef4fa0cb14355b6790530f4b73142: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:17.949873 13212 tablet_bootstrap.cc:492] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8: Bootstrap starting.
I20260811 18:41:17.949893 13213 tablet_bootstrap.cc:492] T 772f94c04ed94b088cc16676c32535f0 P 659ef4fa0cb14355b6790530f4b73142: No bootstrap required, opened a new log
I20260811 18:41:17.950030 13213 ts_tablet_manager.cc:1397] T 772f94c04ed94b088cc16676c32535f0 P 659ef4fa0cb14355b6790530f4b73142: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 18:41:17.950588 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.937709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.30.185.129:32989)
0811 18:41:17.950131 (+ 12422us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:17.950329 (+ 198us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12278,"threads_started":1}
I20260811 18:41:17.950701 13213 raft_consensus.cc:348] T 772f94c04ed94b088cc16676c32535f0 P 659ef4fa0cb14355b6790530f4b73142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34579 } } peers { permanent_uuid: "659ef4fa0cb14355b6790530f4b73142" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42453 } } peers { permanent_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44261 } }
I20260811 18:41:17.950912 13213 raft_consensus.cc:374] T 772f94c04ed94b088cc16676c32535f0 P 659ef4fa0cb14355b6790530f4b73142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:17.951035 13213 raft_consensus.cc:729] T 772f94c04ed94b088cc16676c32535f0 P 659ef4fa0cb14355b6790530f4b73142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 659ef4fa0cb14355b6790530f4b73142, State: Initialized, Role: FOLLOWER
I20260811 18:41:17.951526 13213 consensus_queue.cc:260] T 772f94c04ed94b088cc16676c32535f0 P 659ef4fa0cb14355b6790530f4b73142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34579 } } peers { permanent_uuid: "659ef4fa0cb14355b6790530f4b73142" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42453 } } peers { permanent_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44261 } }
I20260811 18:41:17.951900 13212 tablet_bootstrap.cc:654] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:17.952505 13213 ts_tablet_manager.cc:1428] T 772f94c04ed94b088cc16676c32535f0 P 659ef4fa0cb14355b6790530f4b73142: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 18:41:17.955929 12411 heartbeater.cc:503] Master 127.30.185.188:37311 was elected leader, sending a full tablet report...
I20260811 18:41:17.956854 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.955382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.30.185.129:45133)
0811 18:41:17.955638 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:17.956634 (+ 996us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 18:41:17.957387 13212 tablet_bootstrap.cc:492] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8: No bootstrap required, opened a new log
I20260811 18:41:17.957619 13212 ts_tablet_manager.cc:1397] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260811 18:41:17.958777 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.952776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.30.185.131:45711)
0811 18:41:17.958346 (+ 5570us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:17.958507 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5418,"thread_start_us":5276,"threads_started":1}
I20260811 18:41:17.959636 13212 raft_consensus.cc:348] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34579 } } peers { permanent_uuid: "659ef4fa0cb14355b6790530f4b73142" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42453 } } peers { permanent_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44261 } }
I20260811 18:41:17.959853 13212 raft_consensus.cc:374] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:17.959959 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.959586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.30.185.131:58579)
0811 18:41:17.959737 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:17.959828 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 18:41:17.959955 13212 raft_consensus.cc:729] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce2d62517cd247bd98c6e31ea0f88cc8, State: Initialized, Role: FOLLOWER
I20260811 18:41:17.960232 13212 consensus_queue.cc:260] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34579 } } peers { permanent_uuid: "659ef4fa0cb14355b6790530f4b73142" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42453 } } peers { permanent_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44261 } }
I20260811 18:41:17.960968 13212 ts_tablet_manager.cc:1428] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:41:17.962350 13220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.961388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.30.185.130:39655)
0811 18:41:17.961909 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:17.962106 (+ 197us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":278,"threads_started":1}
I20260811 18:41:17.963388 12726 heartbeater.cc:503] Master 127.30.185.188:37311 was elected leader, sending a full tablet report...
I20260811 18:41:17.963551 13220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:17.962984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.30.185.130:36017)
0811 18:41:17.963282 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:17.963388 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181}
I20260811 18:41:17.987597 13218 raft_consensus.cc:482] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:41:17.987886 13218 raft_consensus.cc:504] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34579 } } peers { permanent_uuid: "659ef4fa0cb14355b6790530f4b73142" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42453 } } peers { permanent_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44261 } }
I20260811 18:41:17.988620 13218 leader_election.cc:288] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b92d06495a8d4bd1a82526fe9071c9d1 (127.30.185.129:34579), 659ef4fa0cb14355b6790530f4b73142 (127.30.185.131:42453)
I20260811 18:41:17.989238 12315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772f94c04ed94b088cc16676c32535f0" candidate_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" is_pre_election: true
I20260811 18:41:17.989351 12913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772f94c04ed94b088cc16676c32535f0" candidate_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "659ef4fa0cb14355b6790530f4b73142" is_pre_election: true
I20260811 18:41:17.989480 12315 raft_consensus.cc:2454] T 772f94c04ed94b088cc16676c32535f0 P b92d06495a8d4bd1a82526fe9071c9d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce2d62517cd247bd98c6e31ea0f88cc8 in term 0.
I20260811 18:41:17.989650 12913 raft_consensus.cc:2454] T 772f94c04ed94b088cc16676c32535f0 P 659ef4fa0cb14355b6790530f4b73142 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce2d62517cd247bd98c6e31ea0f88cc8 in term 0.
I20260811 18:41:17.989944 12492 leader_election.cc:302] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b92d06495a8d4bd1a82526fe9071c9d1, ce2d62517cd247bd98c6e31ea0f88cc8; no voters:
I20260811 18:41:17.990247 13218 raft_consensus.cc:2781] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:41:17.990357 13218 raft_consensus.cc:482] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:41:17.990434 13218 raft_consensus.cc:3037] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:17.992806 13218 raft_consensus.cc:504] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34579 } } peers { permanent_uuid: "659ef4fa0cb14355b6790530f4b73142" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42453 } } peers { permanent_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44261 } }
I20260811 18:41:17.993556 13218 leader_election.cc:288] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8 [CANDIDATE]: Term 1 election: Requested vote from peers b92d06495a8d4bd1a82526fe9071c9d1 (127.30.185.129:34579), 659ef4fa0cb14355b6790530f4b73142 (127.30.185.131:42453)
I20260811 18:41:17.994072 12315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772f94c04ed94b088cc16676c32535f0" candidate_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b92d06495a8d4bd1a82526fe9071c9d1"
I20260811 18:41:17.994328 12315 raft_consensus.cc:3037] T 772f94c04ed94b088cc16676c32535f0 P b92d06495a8d4bd1a82526fe9071c9d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:17.996467 12315 raft_consensus.cc:2454] T 772f94c04ed94b088cc16676c32535f0 P b92d06495a8d4bd1a82526fe9071c9d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce2d62517cd247bd98c6e31ea0f88cc8 in term 1.
I20260811 18:41:17.996912 12492 leader_election.cc:302] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b92d06495a8d4bd1a82526fe9071c9d1, ce2d62517cd247bd98c6e31ea0f88cc8; no voters:
I20260811 18:41:17.996841 12913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772f94c04ed94b088cc16676c32535f0" candidate_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "659ef4fa0cb14355b6790530f4b73142"
I20260811 18:41:17.997079 12913 raft_consensus.cc:3037] T 772f94c04ed94b088cc16676c32535f0 P 659ef4fa0cb14355b6790530f4b73142 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:17.997226 13218 raft_consensus.cc:2781] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:41:17.997728 13218 raft_consensus.cc:686] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 1 LEADER]: Becoming Leader. State: Replica: ce2d62517cd247bd98c6e31ea0f88cc8, State: Running, Role: LEADER
I20260811 18:41:17.998031 13218 consensus_queue.cc:237] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34579 } } peers { permanent_uuid: "659ef4fa0cb14355b6790530f4b73142" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42453 } } peers { permanent_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44261 } }
I20260811 18:41:17.999948 12913 raft_consensus.cc:2454] T 772f94c04ed94b088cc16676c32535f0 P 659ef4fa0cb14355b6790530f4b73142 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce2d62517cd247bd98c6e31ea0f88cc8 in term 1.
I20260811 18:41:18.001196 13220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:18.000528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.30.185.130:54407)
0811 18:41:18.000874 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:18.000994 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209}
I20260811 18:41:18.001652 12050 catalog_manager.cc:5495] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce2d62517cd247bd98c6e31ea0f88cc8 (127.30.185.130). New cstate: current_term: 1 leader_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "659ef4fa0cb14355b6790530f4b73142" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce2d62517cd247bd98c6e31ea0f88cc8" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44261 } health_report { overall_health: HEALTHY } } }
I20260811 18:41:18.024477 13193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:18.020239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35754 (local address 127.30.185.130:44261)
0811 18:41:18.020426 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:18.020431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:18.020448 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:41:18.020593 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:18.020597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:18.020665 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:18.020744 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:41:18.021069 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:18.021814 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:18.023068 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:18.023463 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:18.023557 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:18.023908 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:41:18.024019 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:41:18.024087 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:18.024197 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 18:41:18.024259 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 18:41:18.027899 12315 raft_consensus.cc:1261] T 772f94c04ed94b088cc16676c32535f0 P b92d06495a8d4bd1a82526fe9071c9d1 [term 1 FOLLOWER]: Refusing update from remote peer ce2d62517cd247bd98c6e31ea0f88cc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:41:18.027916 12913 raft_consensus.cc:1261] T 772f94c04ed94b088cc16676c32535f0 P 659ef4fa0cb14355b6790530f4b73142 [term 1 FOLLOWER]: Refusing update from remote peer ce2d62517cd247bd98c6e31ea0f88cc8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:41:18.028592 13221 consensus_queue.cc:1035] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "659ef4fa0cb14355b6790530f4b73142" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:41:18.028952 13218 consensus_queue.cc:1035] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b92d06495a8d4bd1a82526fe9071c9d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:41:18.030395 13012 heartbeater.cc:503] Master 127.30.185.188:37311 was elected leader, sending a full tablet report...
I20260811 18:41:18.032161 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:18.031473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.30.185.129:59993)
0811 18:41:18.031831 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:18.031946 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233}
I20260811 18:41:18.033349 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:18.032695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.30.185.131:54059)
0811 18:41:18.032987 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:18.033115 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260811 18:41:18.039975 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:18.039386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.30.185.131:52743)
0811 18:41:18.039618 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:18.039756 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 18:41:18.045357 13220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:18.044583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.30.185.130:55689)
0811 18:41:18.045058 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:18.045155 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358}
I20260811 18:41:18.046445 13220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:18.045957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.30.185.130:40479)
0811 18:41:18.046153 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:18.046248 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 18:41:18.052304 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:18.045956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.30.185.129:52325)
0811 18:41:18.052096 (+ 6140us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:18.052178 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6021}
I20260811 18:41:18.053566 12047 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:41:18.053834 12047 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:41:18.058933 12047 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eee2ef3d5cd1440eafaa7cd2f3a4fccc: Sending DeleteTablet for 3 replicas of tablet 772f94c04ed94b088cc16676c32535f0
I20260811 18:41:18.060071 12273 tablet_service.cc:1514] Processing DeleteTablet for tablet 772f94c04ed94b088cc16676c32535f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:41:18 UTC) from {username='oryx', principal='oryx/127.30.185.188@KRBTEST.COM'} at 127.0.0.1:33200
I20260811 18:41:18.060225 12582 tablet_service.cc:1514] Processing DeleteTablet for tablet 772f94c04ed94b088cc16676c32535f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:41:18 UTC) from {username='oryx', principal='oryx/127.30.185.188@KRBTEST.COM'} at 127.0.0.1:35742
I20260811 18:41:18.060271 12877 tablet_service.cc:1514] Processing DeleteTablet for tablet 772f94c04ed94b088cc16676c32535f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:41:18 UTC) from {username='oryx', principal='oryx/127.30.185.188@KRBTEST.COM'} at 127.0.0.1:51606
I20260811 18:41:18.061345 13250 tablet_replica.cc:330] T 772f94c04ed94b088cc16676c32535f0 P b92d06495a8d4bd1a82526fe9071c9d1: stopping tablet replica
I20260811 18:41:18.061553 13250 raft_consensus.cc:2229] T 772f94c04ed94b088cc16676c32535f0 P b92d06495a8d4bd1a82526fe9071c9d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:41:18.061882 13250 raft_consensus.cc:2258] T 772f94c04ed94b088cc16676c32535f0 P b92d06495a8d4bd1a82526fe9071c9d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:41:18.062501 13253 tablet_replica.cc:330] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8: stopping tablet replica
I20260811 18:41:18.062561 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 12043
I20260811 18:41:18.062796 13253 raft_consensus.cc:2229] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:41:18.064005 13253 raft_consensus.cc:2258] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:41:18.065199 13253 ts_tablet_manager.cc:1905] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:41:18.071929 13253 ts_tablet_manager.cc:1918] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:41:18.072094 13253 log.cc:1196] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/772f94c04ed94b088cc16676c32535f0
I20260811 18:41:18.072582 13253 ts_tablet_manager.cc:1939] T 772f94c04ed94b088cc16676c32535f0 P ce2d62517cd247bd98c6e31ea0f88cc8: Deleting consensus metadata
I20260811 18:41:18.073997 13259 tablet_replica.cc:330] T 772f94c04ed94b088cc16676c32535f0 P 659ef4fa0cb14355b6790530f4b73142: stopping tablet replica
I20260811 18:41:18.074111 12014 catalog_manager.cc:4841] TS ce2d62517cd247bd98c6e31ea0f88cc8 (127.30.185.130:44261): tablet 772f94c04ed94b088cc16676c32535f0 (table test-table [id=5293ff8d8bbe490e86c9836c56ae3ae9]) successfully deleted
I20260811 18:41:18.074246 13259 raft_consensus.cc:2229] T 772f94c04ed94b088cc16676c32535f0 P 659ef4fa0cb14355b6790530f4b73142 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:41:18.074570 13259 raft_consensus.cc:2258] T 772f94c04ed94b088cc16676c32535f0 P 659ef4fa0cb14355b6790530f4b73142 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:41:18.075380 13250 ts_tablet_manager.cc:1905] T 772f94c04ed94b088cc16676c32535f0 P b92d06495a8d4bd1a82526fe9071c9d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:41:18.082142 13259 ts_tablet_manager.cc:1905] T 772f94c04ed94b088cc16676c32535f0 P 659ef4fa0cb14355b6790530f4b73142: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:41:18.082458 13250 ts_tablet_manager.cc:1918] T 772f94c04ed94b088cc16676c32535f0 P b92d06495a8d4bd1a82526fe9071c9d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:41:18.082597 13250 log.cc:1196] T 772f94c04ed94b088cc16676c32535f0 P b92d06495a8d4bd1a82526fe9071c9d1: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-0/wal/wals/772f94c04ed94b088cc16676c32535f0
I20260811 18:41:18.083009 13250 ts_tablet_manager.cc:1939] T 772f94c04ed94b088cc16676c32535f0 P b92d06495a8d4bd1a82526fe9071c9d1: Deleting consensus metadata
I20260811 18:41:18.084208 12014 catalog_manager.cc:4841] TS b92d06495a8d4bd1a82526fe9071c9d1 (127.30.185.129:34579): tablet 772f94c04ed94b088cc16676c32535f0 (table test-table [id=5293ff8d8bbe490e86c9836c56ae3ae9]) successfully deleted
I20260811 18:41:18.085819 13191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:18.063576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33202 (local address 127.30.185.129:34579)
0811 18:41:18.063800 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:18.063809 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:18.064066 (+ 257us) server_negotiation.cc:407] Connection header received
0811 18:41:18.064257 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:18.064263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:18.064360 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:18.064474 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:18.069360 (+ 4886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:18.070416 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:18.072918 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:18.073571 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:18.073734 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:18.076728 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:18.076745 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:18.076766 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:18.076804 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:18.080106 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:18.080859 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:18.080868 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:18.080870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:18.081013 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:18.083973 (+ 2960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:18.083978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:18.083982 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:18.084483 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:18.084775 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:18.085225 (+ 450us) server_negotiation.cc:299] Negotiation successful
0811 18:41:18.085496 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 18:41:18.086539 12401 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:18.088467 13259 ts_tablet_manager.cc:1918] T 772f94c04ed94b088cc16676c32535f0 P 659ef4fa0cb14355b6790530f4b73142: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:41:18.088624 13259 log.cc:1196] T 772f94c04ed94b088cc16676c32535f0 P 659ef4fa0cb14355b6790530f4b73142: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/772f94c04ed94b088cc16676c32535f0
I20260811 18:41:18.089071 13259 ts_tablet_manager.cc:1939] T 772f94c04ed94b088cc16676c32535f0 P 659ef4fa0cb14355b6790530f4b73142: Deleting consensus metadata
I20260811 18:41:18.090164 12021 catalog_manager.cc:4841] TS 659ef4fa0cb14355b6790530f4b73142 (127.30.185.131:42453): tablet 772f94c04ed94b088cc16676c32535f0 (table test-table [id=5293ff8d8bbe490e86c9836c56ae3ae9]) successfully deleted
I20260811 18:41:18.186761 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 12043
I20260811 18:41:18.212997 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 12440
I20260811 18:41:18.227095 13193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:18.213930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35766 (local address 127.30.185.130:44261)
0811 18:41:18.214130 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:18.214135 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:18.214155 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:41:18.214373 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:18.214379 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:18.214494 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:18.214601 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:18.216073 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:18.216913 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:18.218237 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:18.218638 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:18.218734 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:18.221426 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:18.221450 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:18.221472 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:18.221512 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:18.224292 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:18.224969 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:18.224975 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:18.224978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:18.225112 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:18.225543 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:18.225546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:18.225548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:18.225927 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:18.226150 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:18.226541 (+ 391us) server_negotiation.cc:299] Negotiation successful
0811 18:41:18.226714 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 18:41:18.227841 12720 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:18.364082 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 12440
I20260811 18:41:18.393877 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 12747
I20260811 18:41:18.408588 13192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:18.394643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51610 (local address 127.30.185.131:42453)
0811 18:41:18.394843 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:18.394848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:18.394874 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:41:18.395007 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:18.395011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:18.395076 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:18.395163 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:18.396534 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:18.397433 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:18.398649 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:18.399083 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:18.399182 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:18.402542 (+ 3360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:18.402569 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:18.402592 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:18.402647 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:18.405727 (+ 3080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:18.406563 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:18.406570 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:18.406573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:18.406723 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:18.407050 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:18.407053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:18.407055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:18.407482 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:18.407709 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:18.408111 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 18:41:18.408292 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 18:41:18.409322 13003 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:18.424507 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:18.424069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.0.0.1:42938)
0811 18:41:18.424250 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:18.424340 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 18:41:18.426728 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:18.426140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39547 (local address 127.0.0.1:42954)
0811 18:41:18.426423 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:18.426540 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:39547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 18:41:18.509871 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 12747
I20260811 18:41:18.534265 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 11423
I20260811 18:41:18.535384 11719 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:18.655301 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 11423
I20260811 18:41:18.691484 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 11612
I20260811 18:41:18.692792 12116 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:18.816550 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 11612
2026-08-11T18:41:18Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10431 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:41:18 dist-test-slave-z8x9 krb5kdc[13283](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:41:18 dist-test-slave-z8x9 krb5kdc[13283](info): set up 2 sockets
Aug 11 18:41:18 dist-test-slave-z8x9 krb5kdc[13283](info): commencing operation
krb5kdc: starting...
W20260811 18:41:20.223845 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.352s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:41:20 dist-test-slave-z8x9 krb5kdc[13283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:41:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:41:20Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.190.keytab.
Entry for principal oryx/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.190.keytab.
I20260811 18:41:20.524739 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:40685
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.190.keytab
--principal=oryx/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:33533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:40685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:41:20.868842 13310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:20.869393 13310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:20.869585 13310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:20.877050 13310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:41:20.877244 13310 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 18:41:20.877345 13310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:20.877463 13310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:20.877553 13310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:41:20.877645 13310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:41:20.887930 13310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:33533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:40685
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.190.keytab
--principal=oryx/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:40685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:20.889393 13310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:20.890781 13310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:20.903282 13328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:20.903355 13321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:20.903331 13323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:20.904114 13310 server_base.cc:1034] running on GCE node
I20260811 18:41:20.905036 13310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:20.906605 13310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:20.907876 13310 hybrid_clock.cc:648] HybridClock initialized: now 1786473680907812 us; error 43 us; skew 500 ppm
I20260811 18:41:20.908231 13310 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:20 dist-test-slave-z8x9 krb5kdc[13283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473680, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:20.911649 13310 init.cc:377] Logged in from keytab as oryx/127.30.185.190@KRBTEST.COM (short username oryx)
I20260811 18:41:20.913374 13310 webserver.cc:460] Webserver started at http://127.30.185.190:37027/ using document root <none> and password file <none>
I20260811 18:41:20.914134 13310 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:20.914271 13310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:20.914613 13310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:20.917809 13310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "362393199dbd4999947c30aa9a3d6877"
format_stamp: "Formatted at 2026-08-11 18:41:20 on dist-test-slave-z8x9"
server_key: "a1bb2f0d4f488612ae178fef31751c64"
server_key_iv: "d88dec1d2a4af3e6d6490ef0fe5a1d4f"
server_key_version: "encryptionkey@0"
I20260811 18:41:20.918643 13310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "362393199dbd4999947c30aa9a3d6877"
format_stamp: "Formatted at 2026-08-11 18:41:20 on dist-test-slave-z8x9"
server_key: "a1bb2f0d4f488612ae178fef31751c64"
server_key_iv: "d88dec1d2a4af3e6d6490ef0fe5a1d4f"
server_key_version: "encryptionkey@0"
I20260811 18:41:20.924032 13310 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 18:41:20.927716 13348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:20.929088 13310 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:41:20.929306 13310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "362393199dbd4999947c30aa9a3d6877"
format_stamp: "Formatted at 2026-08-11 18:41:20 on dist-test-slave-z8x9"
server_key: "a1bb2f0d4f488612ae178fef31751c64"
server_key_iv: "d88dec1d2a4af3e6d6490ef0fe5a1d4f"
server_key_version: "encryptionkey@0"
I20260811 18:41:20.929545 13310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:20.954075 13310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:20.958662 13310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:20.958961 13310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:20.981736 13310 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:40685
I20260811 18:41:20.981773 13472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:40685 every 8 connection(s)
I20260811 18:41:20.983119 13310 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:41:20.986884 13475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:20.993422 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 13310
I20260811 18:41:20.993695 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:41:20.994561 31462 external_mini_cluster.cc:1428] Setting key 8b9105276562ac38843da5c51b5f364e
I20260811 18:41:20.998580 13475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877: Bootstrap starting.
I20260811 18:41:21.002351 13475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:21.003521 13475 log.cc:824] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877: Log is configured to *not* fsync() on all Append() calls
I20260811 18:41:21.006827 13475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877: No bootstrap required, opened a new log
Aug 11 18:41:21 dist-test-slave-z8x9 krb5kdc[13283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM
I20260811 18:41:21.015758 13475 raft_consensus.cc:348] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362393199dbd4999947c30aa9a3d6877" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40685 } }
I20260811 18:41:21.016145 13475 raft_consensus.cc:374] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:21.016259 13475 raft_consensus.cc:729] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 362393199dbd4999947c30aa9a3d6877, State: Initialized, Role: FOLLOWER
I20260811 18:41:21.016963 13475 consensus_queue.cc:260] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362393199dbd4999947c30aa9a3d6877" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40685 } }
I20260811 18:41:21.017280 13475 raft_consensus.cc:388] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:41:21.017462 13475 raft_consensus.cc:482] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:41:21.017676 13475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:21.019217 13482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:20.996856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52064 (local address 127.30.185.190:40685)
0811 18:41:20.997770 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:20.997782 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:20.997824 (+ 42us) server_negotiation.cc:407] Connection header received
0811 18:41:20.999614 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:20.999638 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:21.000051 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:21.000391 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:21.002383 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:21.003829 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:21.005451 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:21.006147 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:21.006286 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:21.011298 (+ 5012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:21.011308 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:21.011326 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:21.011372 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:21.014739 (+ 3367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:21.015534 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:21.015541 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:21.015549 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:21.015726 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:21.016178 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:21.016181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:21.016183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:21.016815 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:21.017204 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:21.017672 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 18:41:21.018121 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":461,"threads_started":1}
I20260811 18:41:21.020223 13475 raft_consensus.cc:504] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362393199dbd4999947c30aa9a3d6877" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40685 } }
I20260811 18:41:21.020763 13475 leader_election.cc:302] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 362393199dbd4999947c30aa9a3d6877; no voters:
I20260811 18:41:21.021461 13475 leader_election.cc:288] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:41:21.021624 13488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:41:21.023238 13488 raft_consensus.cc:686] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877 [term 1 LEADER]: Becoming Leader. State: Replica: 362393199dbd4999947c30aa9a3d6877, State: Running, Role: LEADER
I20260811 18:41:21.023521 13475 sys_catalog.cc:564] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:41:21.023813 13488 consensus_queue.cc:237] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362393199dbd4999947c30aa9a3d6877" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40685 } }
I20260811 18:41:21.027202 13491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 362393199dbd4999947c30aa9a3d6877. Latest consensus state: current_term: 1 leader_uuid: "362393199dbd4999947c30aa9a3d6877" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362393199dbd4999947c30aa9a3d6877" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40685 } } }
I20260811 18:41:21.027287 13489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "362393199dbd4999947c30aa9a3d6877" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362393199dbd4999947c30aa9a3d6877" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40685 } } }
I20260811 18:41:21.027612 13491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:21.027691 13489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:21.029381 13506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:41:21.034822 13506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:41:21.045099 13506 catalog_manager.cc:1347] Generated new cluster ID: cc0cc0009eba4909b42d1f03e1f06d5f
I20260811 18:41:21.045321 13506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:41:21.058377 13506 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:41:21.060323 13506 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:41:21.070303 13506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877: Generated new TSK 0
I20260811 18:41:21.071429 13506 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.185.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.185.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.185.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.129.keytab.
Entry for principal oryx/127.30.185.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.129.keytab.
I20260811 18:41:21.145262 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:40685
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.129.keytab
--principal=oryx/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:33533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:41:21.499794 13523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:21.500095 13523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:21.500204 13523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:21.507632 13523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:21.507772 13523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:21.507889 13523 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:41:21.507977 13523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:41:21.518303 13523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:33533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.129.keytab
--principal=oryx/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.185.190:40685
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:21.519693 13523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:21.521099 13523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:21.535202 13533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:21.535234 13534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:21.535171 13539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:21.536309 13523 server_base.cc:1034] running on GCE node
I20260811 18:41:21.536978 13523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:21.537997 13523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:21.539214 13523 hybrid_clock.cc:648] HybridClock initialized: now 1786473681539113 us; error 72 us; skew 500 ppm
I20260811 18:41:21.539489 13523 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:21 dist-test-slave-z8x9 krb5kdc[13283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473681, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:21.546797 13523 init.cc:377] Logged in from keytab as oryx/127.30.185.129@KRBTEST.COM (short username oryx)
I20260811 18:41:21.548223 13523 webserver.cc:460] Webserver started at http://127.30.185.129:43973/ using document root <none> and password file <none>
I20260811 18:41:21.548915 13523 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:21.549005 13523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:21.549355 13523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:21.552326 13523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "3e6e59471bcd4c31b213dfe952991ec4"
format_stamp: "Formatted at 2026-08-11 18:41:21 on dist-test-slave-z8x9"
server_key: "cca9572a43cbc6d2ccf47c0997726860"
server_key_iv: "6cd8c049231d979d284b8647e4f08d67"
server_key_version: "encryptionkey@0"
I20260811 18:41:21.553153 13523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "3e6e59471bcd4c31b213dfe952991ec4"
format_stamp: "Formatted at 2026-08-11 18:41:21 on dist-test-slave-z8x9"
server_key: "cca9572a43cbc6d2ccf47c0997726860"
server_key_iv: "6cd8c049231d979d284b8647e4f08d67"
server_key_version: "encryptionkey@0"
I20260811 18:41:21.558511 13523 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 18:41:21.561896 13556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:21.563166 13523 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 18:41:21.563303 13523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "3e6e59471bcd4c31b213dfe952991ec4"
format_stamp: "Formatted at 2026-08-11 18:41:21 on dist-test-slave-z8x9"
server_key: "cca9572a43cbc6d2ccf47c0997726860"
server_key_iv: "6cd8c049231d979d284b8647e4f08d67"
server_key_version: "encryptionkey@0"
I20260811 18:41:21.563504 13523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:21.583456 13523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:21.588049 13523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:21.588286 13523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:21.591004 13523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:21.591123 13523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:21.591264 13523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:21.591342 13523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:41:21 dist-test-slave-z8x9 krb5kdc[13283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473681, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM
I20260811 18:41:21.618250 13576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:21.591784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:40573 (local address 127.30.185.190:40685)
0811 18:41:21.592383 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:21.592387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:21.593542 (+ 1155us) server_negotiation.cc:407] Connection header received
0811 18:41:21.595595 (+ 2053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:21.595600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:21.595697 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:21.595822 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:21.598194 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:21.599327 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:21.602083 (+ 2756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:21.602933 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:21.603076 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:21.609144 (+ 6068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:21.609154 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:21.609162 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:21.609240 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:21.613247 (+ 4007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:21.614598 (+ 1351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:21.614606 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:21.614610 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:21.614779 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:21.615574 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:21.615579 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:21.615581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:21.616013 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:21.616175 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:21.617366 (+ 1191us) server_negotiation.cc:299] Negotiation successful
0811 18:41:21.617695 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":367,"threads_started":1}
I20260811 18:41:21.618916 13577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:21.592463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:40685 (local address 127.30.185.129:40573)
0811 18:41:21.593366 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:21.593415 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:21.595304 (+ 1889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:21.596084 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:21.596099 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:21.596955 (+ 856us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:21.597953 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:21.597985 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:21.599577 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:21.599583 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:21.601904 (+ 2321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:21.601926 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:21.603165 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:21.603171 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:21.603445 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:21.604395 (+ 950us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:21.604445 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:21.608761 (+ 4316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:21.613489 (+ 4728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:21.613504 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:21.613532 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:21.614410 (+ 878us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:21.614908 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:21.614917 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:21.614921 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:21.615302 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:21.616150 (+ 848us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:21.616159 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:21.616772 (+ 613us) client_negotiation.cc:769] Sending connection context
0811 18:41:21.617152 (+ 380us) client_negotiation.cc:240] Negotiation successful
0811 18:41:21.617658 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":363,"threads_started":1}
I20260811 18:41:21.642630 13758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:21.626599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:40685 (local address 127.30.185.129:48095)
0811 18:41:21.627221 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:21.627236 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:21.627404 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:21.627720 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:21.627724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:21.628196 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:21.628504 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:21.628513 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:21.629775 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:21.629781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:21.632072 (+ 2291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:21.632096 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:21.633181 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:21.633188 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:21.633554 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:21.634569 (+ 1015us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:21.634592 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:21.635458 (+ 866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:21.639105 (+ 3647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:21.639114 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:21.639120 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:21.639724 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:21.640218 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:21.640222 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:21.640226 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:21.640411 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:21.641136 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:21.641143 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:21.641456 (+ 313us) client_negotiation.cc:769] Sending connection context
0811 18:41:21.641656 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 18:41:21.642122 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":338,"threads_started":1}
I20260811 18:41:21.642768 13576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:21.626496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:48095 (local address 127.30.185.190:40685)
0811 18:41:21.626679 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:21.626683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:21.627332 (+ 649us) server_negotiation.cc:407] Connection header received
0811 18:41:21.627526 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:21.627531 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:21.627594 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:21.627694 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:21.628650 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:21.629568 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:21.632259 (+ 2691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:21.632911 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:21.633065 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:21.635683 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:21.635689 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:21.635692 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:21.635724 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:21.638953 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:21.639874 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:21.639880 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:21.639884 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:21.640066 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:21.640544 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:21.640550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:21.640556 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:21.640993 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:21.641178 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:21.642164 (+ 986us) server_negotiation.cc:299] Negotiation successful
0811 18:41:21.642406 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 18:41:21.649927 13523 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:37959
I20260811 18:41:21.649974 13817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:37959 every 8 connection(s)
I20260811 18:41:21.651217 13523 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:41:21.653385 13818 heartbeater.cc:347] Connected to a master server at 127.30.185.190:40685
I20260811 18:41:21.653631 13818 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:21.654306 13818 heartbeater.cc:511] Master 127.30.185.190:40685 requested a full tablet report, sending...
I20260811 18:41:21.656292 13381 ts_manager.cc:194] Registered new tserver with Master: 3e6e59471bcd4c31b213dfe952991ec4 (127.30.185.129:37959)
I20260811 18:41:21.656744 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 13523
I20260811 18:41:21.656975 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:41:21.657825 31462 external_mini_cluster.cc:1428] Setting key e6837d0069e1ecf8e6de5623bd58424a
I20260811 18:41:21.658592 13381 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:40573
WARNING: no policy specified for oryx/127.30.185.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.185.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.185.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.130.keytab.
Entry for principal oryx/127.30.185.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.130.keytab.
I20260811 18:41:21.720850 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:40685
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.130.keytab
--principal=oryx/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:33533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:41:22.067158 13826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:22.067447 13826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:22.067641 13826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:22.075302 13826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:22.075418 13826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:22.075577 13826 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:41:22.075712 13826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:41:22.085711 13826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:33533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.130.keytab
--principal=oryx/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.185.190:40685
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:22.087030 13826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:22.088380 13826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:22.102029 13843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:22.102090 13844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:22.101998 13847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:22.103273 13826 server_base.cc:1034] running on GCE node
I20260811 18:41:22.104048 13826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:22.105106 13826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:22.106422 13826 hybrid_clock.cc:648] HybridClock initialized: now 1786473682106346 us; error 42 us; skew 500 ppm
I20260811 18:41:22.106751 13826 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:22 dist-test-slave-z8x9 krb5kdc[13283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473682, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:22.110353 13826 init.cc:377] Logged in from keytab as oryx/127.30.185.130@KRBTEST.COM (short username oryx)
I20260811 18:41:22.112236 13826 webserver.cc:460] Webserver started at http://127.30.185.130:40193/ using document root <none> and password file <none>
I20260811 18:41:22.113024 13826 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:22.113199 13826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:22.113561 13826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:22.116595 13826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "56dd0faaf36046478b6dfddbe800f300"
format_stamp: "Formatted at 2026-08-11 18:41:22 on dist-test-slave-z8x9"
server_key: "14579ecb88dc7f56645ff8a2141f6848"
server_key_iv: "8c35005e87779f58cd2831cb3fb1ff0e"
server_key_version: "encryptionkey@0"
I20260811 18:41:22.117496 13826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "56dd0faaf36046478b6dfddbe800f300"
format_stamp: "Formatted at 2026-08-11 18:41:22 on dist-test-slave-z8x9"
server_key: "14579ecb88dc7f56645ff8a2141f6848"
server_key_iv: "8c35005e87779f58cd2831cb3fb1ff0e"
server_key_version: "encryptionkey@0"
I20260811 18:41:22.123356 13826 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:41:22.127121 13858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:22.128600 13826 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:41:22.128816 13826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "56dd0faaf36046478b6dfddbe800f300"
format_stamp: "Formatted at 2026-08-11 18:41:22 on dist-test-slave-z8x9"
server_key: "14579ecb88dc7f56645ff8a2141f6848"
server_key_iv: "8c35005e87779f58cd2831cb3fb1ff0e"
server_key_version: "encryptionkey@0"
I20260811 18:41:22.129086 13826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:22.150288 13826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:22.154968 13826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:22.155251 13826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:22.158114 13826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:22.158200 13826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:22.158353 13826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:22.158412 13826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:41:22 dist-test-slave-z8x9 krb5kdc[13283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473682, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM
I20260811 18:41:22.186576 13877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:22.159432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:40685 (local address 127.30.185.130:34905)
0811 18:41:22.160381 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:22.160433 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:22.162475 (+ 2042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:22.163288 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:22.163297 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:22.164076 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:22.165131 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:22.165197 (+ 66us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.166778 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.166787 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:22.169794 (+ 3007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:22.169812 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.170968 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.170979 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:22.171414 (+ 435us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:22.172395 (+ 981us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:22.172441 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:22.176463 (+ 4022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:22.180833 (+ 4370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:22.180851 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:22.180877 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:22.181702 (+ 825us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:22.182246 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:22.182251 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:22.182255 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:22.182515 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:22.183240 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:22.183257 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:22.183681 (+ 424us) client_negotiation.cc:769] Sending connection context
0811 18:41:22.184009 (+ 328us) client_negotiation.cc:240] Negotiation successful
0811 18:41:22.184779 (+ 770us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":426,"threads_started":1}
I20260811 18:41:22.187760 13873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:22.158686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:34905 (local address 127.30.185.190:40685)
0811 18:41:22.159375 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:22.159384 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:22.160575 (+ 1191us) server_negotiation.cc:407] Connection header received
0811 18:41:22.162793 (+ 2218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:22.162799 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:22.162897 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:22.163019 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:22.165388 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.166550 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.170021 (+ 3471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.170664 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.170777 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:22.176773 (+ 5996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:22.176779 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:22.176783 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:22.176822 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:22.180583 (+ 3761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:22.181896 (+ 1313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:22.181900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:22.181902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:22.182078 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:22.182706 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:22.182714 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:22.182720 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:22.183098 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:22.183272 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:22.187221 (+ 3949us) server_negotiation.cc:299] Negotiation successful
0811 18:41:22.187453 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":412,"threads_started":1}
I20260811 18:41:22.208590 13826 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:40829
I20260811 18:41:22.208621 14105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:40829 every 8 connection(s)
I20260811 18:41:22.210027 13826 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:41:22.210477 13873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:22.194048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:33871 (local address 127.30.185.190:40685)
0811 18:41:22.194266 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:22.194273 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:22.195000 (+ 727us) server_negotiation.cc:407] Connection header received
0811 18:41:22.195263 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:22.195269 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:22.195354 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:22.195467 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:22.196557 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.197965 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.200093 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.201012 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.201225 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:22.203108 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:22.203118 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:22.203127 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:22.203179 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:22.207325 (+ 4146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:22.208251 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:22.208260 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:22.208266 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:22.208443 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:22.208895 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:22.208903 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:22.208907 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:22.209322 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:22.209478 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:22.209887 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 18:41:22.210118 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 18:41:22.210618 14057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:22.194066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:40685 (local address 127.30.185.130:33871)
0811 18:41:22.194858 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:22.194906 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:22.195089 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:22.195489 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:22.195492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:22.195965 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:22.196367 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:22.196383 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.198127 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.198135 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:22.199907 (+ 1772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:22.199928 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.201301 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.201306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:22.201422 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:22.202121 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:22.202146 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:22.202891 (+ 745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:22.207476 (+ 4585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:22.207481 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:22.207485 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:22.208102 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:22.208571 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:22.208579 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:22.208585 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:22.208782 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:22.209440 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:22.209445 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:22.209634 (+ 189us) client_negotiation.cc:769] Sending connection context
0811 18:41:22.209788 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 18:41:22.210155 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":452,"threads_started":1}
I20260811 18:41:22.212288 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 13826
I20260811 18:41:22.212492 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:41:22.212881 31462 external_mini_cluster.cc:1428] Setting key 3e7db4e1a2f6557c4e75d2883e354262
I20260811 18:41:22.214707 14109 heartbeater.cc:347] Connected to a master server at 127.30.185.190:40685
I20260811 18:41:22.215309 14109 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:22.215962 14109 heartbeater.cc:511] Master 127.30.185.190:40685 requested a full tablet report, sending...
I20260811 18:41:22.217376 13381 ts_manager.cc:194] Registered new tserver with Master: 56dd0faaf36046478b6dfddbe800f300 (127.30.185.130:40829)
I20260811 18:41:22.218180 13381 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:34905
WARNING: no policy specified for oryx/127.30.185.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.185.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.185.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.131.keytab.
Entry for principal oryx/127.30.185.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.131.keytab.
I20260811 18:41:22.275997 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:40685
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.131.keytab
--principal=oryx/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:33533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:41:22.618000 14114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:22.618302 14114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:22.618476 14114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:22.625999 14114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:22.626114 14114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:22.626255 14114 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:41:22.626350 14114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:41:22.636715 14114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:33533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.131.keytab
--principal=oryx/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.185.190:40685
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:22.638077 14114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:22.639447 14114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:22.654271 14121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:22.654289 14122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:22.654343 14124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:22.655202 14114 server_base.cc:1034] running on GCE node
I20260811 18:41:22.655879 14114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:22.656765 14114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:22.658070 14114 hybrid_clock.cc:648] HybridClock initialized: now 1786473682657986 us; error 53 us; skew 500 ppm
I20260811 18:41:22.658398 14114 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:22 dist-test-slave-z8x9 krb5kdc[13283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473682, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:22.662155 13818 heartbeater.cc:503] Master 127.30.185.190:40685 was elected leader, sending a full tablet report...
I20260811 18:41:22.662698 14114 init.cc:377] Logged in from keytab as oryx/127.30.185.131@KRBTEST.COM (short username oryx)
I20260811 18:41:22.664692 14114 webserver.cc:460] Webserver started at http://127.30.185.131:40461/ using document root <none> and password file <none>
I20260811 18:41:22.665618 14114 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:22.665756 14114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:22.666136 14114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:22.670156 14114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "c36cf4f91b244ca2b5b6f8f003a5a033"
format_stamp: "Formatted at 2026-08-11 18:41:22 on dist-test-slave-z8x9"
server_key: "8ab7197cebb54122020d38900d3bdc8e"
server_key_iv: "ae4ca876e05b9d52eac36b0898df45d2"
server_key_version: "encryptionkey@0"
I20260811 18:41:22.671150 14114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "c36cf4f91b244ca2b5b6f8f003a5a033"
format_stamp: "Formatted at 2026-08-11 18:41:22 on dist-test-slave-z8x9"
server_key: "8ab7197cebb54122020d38900d3bdc8e"
server_key_iv: "ae4ca876e05b9d52eac36b0898df45d2"
server_key_version: "encryptionkey@0"
I20260811 18:41:22.676326 14114 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 18:41:22.679705 14141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:22.681025 14114 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:41:22.681226 14114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "c36cf4f91b244ca2b5b6f8f003a5a033"
format_stamp: "Formatted at 2026-08-11 18:41:22 on dist-test-slave-z8x9"
server_key: "8ab7197cebb54122020d38900d3bdc8e"
server_key_iv: "ae4ca876e05b9d52eac36b0898df45d2"
server_key_version: "encryptionkey@0"
I20260811 18:41:22.681460 14114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:22.702178 14114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:22.706359 14114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:22.706647 14114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:22.709385 14114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:22.709494 14114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:22.709647 14114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:22.709722 14114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:41:22 dist-test-slave-z8x9 krb5kdc[13283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473682, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM
I20260811 18:41:22.733141 13873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:22.710089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:52009 (local address 127.30.185.190:40685)
0811 18:41:22.710344 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:22.710353 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:22.711674 (+ 1321us) server_negotiation.cc:407] Connection header received
0811 18:41:22.713822 (+ 2148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:22.713828 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:22.713920 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:22.714036 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:22.716291 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.717079 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.719601 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.720240 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.720402 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:22.725339 (+ 4937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:22.725346 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:22.725351 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:22.725396 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:22.729149 (+ 3753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:22.730082 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:22.730089 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:22.730093 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:22.730240 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:22.730854 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:22.730862 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:22.730867 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:22.731348 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:22.731558 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:22.732265 (+ 707us) server_negotiation.cc:299] Negotiation successful
0811 18:41:22.732580 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:41:22.733995 14164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:22.710621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:40685 (local address 127.30.185.131:52009)
0811 18:41:22.711505 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:22.711551 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:22.713529 (+ 1978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:22.714300 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:22.714309 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:22.715129 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:22.716126 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:22.716145 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.717290 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.717295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:22.719446 (+ 2151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:22.719459 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.720600 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.720615 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:22.720963 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:22.722013 (+ 1050us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:22.722042 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:22.725015 (+ 2973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:22.729325 (+ 4310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:22.729334 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:22.729355 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:22.729938 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:22.730355 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:22.730360 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:22.730371 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:22.730708 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:22.731502 (+ 794us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:22.731508 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:22.731855 (+ 347us) client_negotiation.cc:769] Sending connection context
0811 18:41:22.732104 (+ 249us) client_negotiation.cc:240] Negotiation successful
0811 18:41:22.732505 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":391,"threads_started":1}
I20260811 18:41:22.757951 13873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:22.739953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:40561 (local address 127.30.185.190:40685)
0811 18:41:22.740152 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:22.740160 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:22.741236 (+ 1076us) server_negotiation.cc:407] Connection header received
0811 18:41:22.741467 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:22.741474 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:22.741565 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:22.741680 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:22.743265 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.744007 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.746634 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.747069 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.747174 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:22.750224 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:22.750229 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:22.750233 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:22.750264 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:22.754201 (+ 3937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:22.755175 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:22.755184 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:22.755190 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:22.755390 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:22.755904 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:22.755913 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:22.755916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:22.756341 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:22.756501 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:22.757385 (+ 884us) server_negotiation.cc:299] Negotiation successful
0811 18:41:22.757618 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 18:41:22.758013 14273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:22.740177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:40685 (local address 127.30.185.131:40561)
0811 18:41:22.741108 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:22.741139 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:22.741308 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:22.741733 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:22.741746 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:22.742587 (+ 841us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:22.743078 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:22.743095 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.744149 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.744157 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:22.746381 (+ 2224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:22.746402 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.747282 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.747291 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:22.747563 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:22.748429 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:22.748461 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:22.750018 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:22.754363 (+ 4345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:22.754369 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:22.754373 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:22.755006 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:22.755526 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:22.755530 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:22.755532 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:22.755767 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:22.756573 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:22.756576 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:22.756851 (+ 275us) client_negotiation.cc:769] Sending connection context
0811 18:41:22.757059 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 18:41:22.757483 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"thread_start_us":513,"threads_started":1}
I20260811 18:41:22.768287 14114 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:37899
I20260811 18:41:22.768328 14379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:37899 every 8 connection(s)
I20260811 18:41:22.769579 14114 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:41:22.771621 14380 heartbeater.cc:347] Connected to a master server at 127.30.185.190:40685
I20260811 18:41:22.771828 14380 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:22.772467 14380 heartbeater.cc:511] Master 127.30.185.190:40685 requested a full tablet report, sending...
I20260811 18:41:22.773743 13381 ts_manager.cc:194] Registered new tserver with Master: c36cf4f91b244ca2b5b6f8f003a5a033 (127.30.185.131:37899)
I20260811 18:41:22.774538 13381 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:52009
I20260811 18:41:22.775938 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 14114
I20260811 18:41:22.776145 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:41:22.776696 31462 external_mini_cluster.cc:1428] Setting key a09d3356c19f6b08282712ba2711f6a4
I20260811 18:41:22.779634 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 18:41:22 dist-test-slave-z8x9 krb5kdc[13283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473682, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:41:22 dist-test-slave-z8x9 krb5kdc[13283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:22 dist-test-slave-z8x9 krb5kdc[13283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 18:41:22.845820 13873 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:22.834279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48796 (local address 127.30.185.190:40685)
0811 18:41:22.834491 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:22.834498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:22.834786 (+ 288us) server_negotiation.cc:407] Connection header received
0811 18:41:22.834947 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:22.834952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:22.835025 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:22.835129 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:22.836949 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.837865 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.839112 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.839530 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.839640 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:22.845506 (+ 5866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48796: BlockingRecv error: recv got EOF from 127.0.0.1:48796 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 18:41:22 dist-test-slave-z8x9 krb5kdc[13283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473682, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM
I20260811 18:41:22.863615 13873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:22.848916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48808 (local address 127.30.185.190:40685)
0811 18:41:22.849165 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:22.849202 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:22.849459 (+ 257us) server_negotiation.cc:407] Connection header received
0811 18:41:22.849656 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:22.849661 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:22.849747 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:22.849841 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:22.851014 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.851763 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.853059 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.853604 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.853709 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:22.857824 (+ 4115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:22.857830 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:22.857835 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:22.857874 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:22.860799 (+ 2925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:22.861605 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:22.861610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:22.861612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:22.861784 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:22.862258 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:22.862263 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:22.862265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:22.862573 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:22.862803 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:22.863070 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 18:41:22.863255 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 18:41:22.868669 13381 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:41:22.876160 13381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:41:22.913327 14418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:22.890233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49974 (local address 127.30.185.130:40829)
0811 18:41:22.902456 (+ 12223us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:22.902470 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:22.903844 (+ 1374us) server_negotiation.cc:407] Connection header received
0811 18:41:22.904387 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:22.904398 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:22.904709 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:22.904901 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:22.906034 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.907594 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.910383 (+ 2789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.911933 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.912117 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:22.912309 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:22.912771 (+ 462us) server_negotiation.cc:299] Negotiation successful
0811 18:41:22.912984 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11248,"thread_start_us":11766,"threads_started":1}
I20260811 18:41:22.913404 14416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:22.890048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:40829 (local address 127.0.0.1:49974)
0811 18:41:22.903635 (+ 13587us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:22.903692 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:22.904057 (+ 365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:22.905303 (+ 1246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:22.905316 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:22.905361 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:22.905860 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:22.905874 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.907782 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.907787 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:22.910146 (+ 2359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:22.910162 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.912184 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.912192 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:22.912470 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:22.912538 (+ 68us) client_negotiation.cc:769] Sending connection context
0811 18:41:22.912672 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 18:41:22.912878 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13319,"mutex_wait_us":1,"thread_start_us":13162,"threads_started":1}
I20260811 18:41:22.915429 14415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:22.890072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:37899 (local address 127.0.0.1:52884)
0811 18:41:22.906516 (+ 16444us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:22.906541 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:22.906672 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:22.907356 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:22.907375 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:22.907402 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:22.907829 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:22.907847 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.910126 (+ 2279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.910132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:22.912244 (+ 2112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:22.912258 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.914676 (+ 2418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.914683 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:22.914867 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:22.914905 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 18:41:22.915007 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 18:41:22.915132 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16276,"thread_start_us":16114,"threads_started":1}
I20260811 18:41:22.915921 14425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:22.890755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:37959 (local address 127.0.0.1:54410)
0811 18:41:22.906937 (+ 16182us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:22.906959 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:22.907187 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:22.908057 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:22.908061 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:22.908085 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:22.908579 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:22.908598 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.910508 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.910513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:22.912806 (+ 2293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:22.912820 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.915064 (+ 2244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.915072 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:22.915251 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:22.915288 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 18:41:22.915402 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 18:41:22.915536 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16068,"mutex_wait_us":1,"thread_start_us":15906,"threads_started":1}
I20260811 18:41:22.916021 14428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:22.891539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54410 (local address 127.30.185.129:37959)
0811 18:41:22.903035 (+ 11496us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:22.903046 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:22.907082 (+ 4036us) server_negotiation.cc:407] Connection header received
0811 18:41:22.907399 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:22.907407 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:22.907628 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:22.907796 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:22.908785 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.910324 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.912994 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.914526 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.914674 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:22.914840 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:22.915466 (+ 626us) server_negotiation.cc:299] Negotiation successful
0811 18:41:22.915660 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11286,"thread_start_us":11157,"threads_started":1}
I20260811 18:41:22.917404 14423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:22.890675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52884 (local address 127.30.185.131:37899)
0811 18:41:22.905985 (+ 15310us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:22.905998 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:22.906815 (+ 817us) server_negotiation.cc:407] Connection header received
0811 18:41:22.906913 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:22.906920 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:22.907154 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:22.908083 (+ 929us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:22.908463 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.909733 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.912465 (+ 2732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:22.914143 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:22.914337 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:22.914530 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:22.915918 (+ 1388us) server_negotiation.cc:299] Negotiation successful
0811 18:41:22.916995 (+ 1077us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15075,"thread_start_us":14847,"threads_started":1}
I20260811 18:41:22.918274 13978 tablet_service.cc:1467] Processing CreateTablet for tablet d5627b3336d846b089f8df2ba997f15c (DEFAULT_TABLE table=test-table [id=e5d8f464574b4d16a88d3bde47283717]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:41:22.919934 13978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5627b3336d846b089f8df2ba997f15c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:22.919535 13675 tablet_service.cc:1467] Processing CreateTablet for tablet d5627b3336d846b089f8df2ba997f15c (DEFAULT_TABLE table=test-table [id=e5d8f464574b4d16a88d3bde47283717]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:41:22.920915 13675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5627b3336d846b089f8df2ba997f15c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:22.921164 14252 tablet_service.cc:1467] Processing CreateTablet for tablet d5627b3336d846b089f8df2ba997f15c (DEFAULT_TABLE table=test-table [id=e5d8f464574b4d16a88d3bde47283717]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:41:22.922586 14252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5627b3336d846b089f8df2ba997f15c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:22.933542 14429 tablet_bootstrap.cc:492] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4: Bootstrap starting.
I20260811 18:41:22.937299 14429 tablet_bootstrap.cc:654] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:22.938023 14432 tablet_bootstrap.cc:492] T d5627b3336d846b089f8df2ba997f15c P c36cf4f91b244ca2b5b6f8f003a5a033: Bootstrap starting.
I20260811 18:41:22.939414 14429 log.cc:824] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4: Log is configured to *not* fsync() on all Append() calls
I20260811 18:41:22.939725 14430 tablet_bootstrap.cc:492] T d5627b3336d846b089f8df2ba997f15c P 56dd0faaf36046478b6dfddbe800f300: Bootstrap starting.
I20260811 18:41:22.942806 14430 tablet_bootstrap.cc:654] T d5627b3336d846b089f8df2ba997f15c P 56dd0faaf36046478b6dfddbe800f300: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:22.942904 14432 tablet_bootstrap.cc:654] T d5627b3336d846b089f8df2ba997f15c P c36cf4f91b244ca2b5b6f8f003a5a033: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:22.943511 14429 tablet_bootstrap.cc:492] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4: No bootstrap required, opened a new log
I20260811 18:41:22.943825 14429 ts_tablet_manager.cc:1397] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260811 18:41:22.943912 14430 log.cc:824] T d5627b3336d846b089f8df2ba997f15c P 56dd0faaf36046478b6dfddbe800f300: Log is configured to *not* fsync() on all Append() calls
I20260811 18:41:22.944154 14432 log.cc:824] T d5627b3336d846b089f8df2ba997f15c P c36cf4f91b244ca2b5b6f8f003a5a033: Log is configured to *not* fsync() on all Append() calls
I20260811 18:41:22.947481 14432 tablet_bootstrap.cc:492] T d5627b3336d846b089f8df2ba997f15c P c36cf4f91b244ca2b5b6f8f003a5a033: No bootstrap required, opened a new log
I20260811 18:41:22.947547 14430 tablet_bootstrap.cc:492] T d5627b3336d846b089f8df2ba997f15c P 56dd0faaf36046478b6dfddbe800f300: No bootstrap required, opened a new log
I20260811 18:41:22.947789 14432 ts_tablet_manager.cc:1397] T d5627b3336d846b089f8df2ba997f15c P c36cf4f91b244ca2b5b6f8f003a5a033: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 18:41:22.947851 14430 ts_tablet_manager.cc:1397] T d5627b3336d846b089f8df2ba997f15c P 56dd0faaf36046478b6dfddbe800f300: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.008s
I20260811 18:41:22.954924 14429 raft_consensus.cc:348] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } } peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } }
I20260811 18:41:22.955446 14429 raft_consensus.cc:374] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:22.955549 14429 raft_consensus.cc:729] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e6e59471bcd4c31b213dfe952991ec4, State: Initialized, Role: FOLLOWER
I20260811 18:41:22.955592 14432 raft_consensus.cc:348] T d5627b3336d846b089f8df2ba997f15c P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } } peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } }
I20260811 18:41:22.956069 14432 raft_consensus.cc:374] T d5627b3336d846b089f8df2ba997f15c P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:22.956187 14432 raft_consensus.cc:729] T d5627b3336d846b089f8df2ba997f15c P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36cf4f91b244ca2b5b6f8f003a5a033, State: Initialized, Role: FOLLOWER
I20260811 18:41:22.956298 14429 consensus_queue.cc:260] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } } peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } }
I20260811 18:41:22.956776 14432 consensus_queue.cc:260] T d5627b3336d846b089f8df2ba997f15c P c36cf4f91b244ca2b5b6f8f003a5a033 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } } peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } }
I20260811 18:41:22.958313 14429 ts_tablet_manager.cc:1428] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4: Time spent starting tablet: real 0.014s user 0.015s sys 0.001s
I20260811 18:41:22.958514 14430 raft_consensus.cc:348] T d5627b3336d846b089f8df2ba997f15c P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } } peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } }
I20260811 18:41:22.958967 14430 raft_consensus.cc:374] T d5627b3336d846b089f8df2ba997f15c P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:22.959082 14430 raft_consensus.cc:729] T d5627b3336d846b089f8df2ba997f15c P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56dd0faaf36046478b6dfddbe800f300, State: Initialized, Role: FOLLOWER
I20260811 18:41:22.959606 14380 heartbeater.cc:503] Master 127.30.185.190:40685 was elected leader, sending a full tablet report...
I20260811 18:41:22.959787 14430 consensus_queue.cc:260] T d5627b3336d846b089f8df2ba997f15c P 56dd0faaf36046478b6dfddbe800f300 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } } peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } }
I20260811 18:41:22.960178 14432 ts_tablet_manager.cc:1428] T d5627b3336d846b089f8df2ba997f15c P c36cf4f91b244ca2b5b6f8f003a5a033: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260811 18:41:22.961150 14109 heartbeater.cc:503] Master 127.30.185.190:40685 was elected leader, sending a full tablet report...
I20260811 18:41:22.961802 14430 ts_tablet_manager.cc:1428] T d5627b3336d846b089f8df2ba997f15c P 56dd0faaf36046478b6dfddbe800f300: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
W20260811 18:41:22.963689 14111 tablet.cc:2369] T d5627b3336d846b089f8df2ba997f15c P 56dd0faaf36046478b6dfddbe800f300: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:41:23.011284 14439 raft_consensus.cc:482] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:41:23.011615 14439 raft_consensus.cc:504] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } } peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } }
I20260811 18:41:23.013533 14439 leader_election.cc:288] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56dd0faaf36046478b6dfddbe800f300 (127.30.185.130:40829), c36cf4f91b244ca2b5b6f8f003a5a033 (127.30.185.131:37899)
I20260811 18:41:23.021420 14418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:23.013942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:47263 (local address 127.30.185.130:40829)
0811 18:41:23.014177 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:23.014185 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:23.015168 (+ 983us) server_negotiation.cc:407] Connection header received
0811 18:41:23.015458 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:23.015464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:23.015548 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:23.015617 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:23.016427 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.017354 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.019336 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.020270 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.020402 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:23.020455 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:23.020891 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 18:41:23.021026 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 18:41:23.021484 14443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:23.014006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:40829 (local address 127.30.185.129:47263)
0811 18:41:23.015048 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:23.015063 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:23.015239 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:23.015840 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:23.015845 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:23.015871 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:23.016266 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:23.016277 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.017481 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.017486 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:23.019180 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:23.019193 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.020481 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.020488 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:23.020658 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:23.020705 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 18:41:23.020816 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 18:41:23.020965 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":911,"mutex_wait_us":25,"thread_start_us":731,"threads_started":1}
W20260811 18:41:23.021458 14382 tablet.cc:2369] T d5627b3336d846b089f8df2ba997f15c P c36cf4f91b244ca2b5b6f8f003a5a033: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:41:23.021878 14423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:23.014176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:58325 (local address 127.30.185.131:37899)
0811 18:41:23.014461 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:23.014471 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:23.015737 (+ 1266us) server_negotiation.cc:407] Connection header received
0811 18:41:23.015901 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:23.015906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:23.015994 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:23.016102 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:23.016861 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.017823 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.019749 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.020645 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.020821 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:23.020893 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:23.021318 (+ 425us) server_negotiation.cc:299] Negotiation successful
0811 18:41:23.021484 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 18:41:23.022451 14445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:23.014107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:37899 (local address 127.30.185.129:58325)
0811 18:41:23.015620 (+ 1513us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:23.015637 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:23.015763 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:23.016124 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:23.016128 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:23.016162 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:23.016622 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:23.016635 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.017965 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.017968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:23.019579 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:23.019594 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.020894 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.020901 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:23.021085 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:23.021124 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 18:41:23.021837 (+ 713us) client_negotiation.cc:240] Negotiation successful
0811 18:41:23.022013 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1376,"mutex_wait_us":31,"thread_start_us":1218,"threads_started":1}
I20260811 18:41:23.022773 14022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5627b3336d846b089f8df2ba997f15c" candidate_uuid: "3e6e59471bcd4c31b213dfe952991ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56dd0faaf36046478b6dfddbe800f300" is_pre_election: true
I20260811 18:41:23.023345 14022 raft_consensus.cc:2454] T d5627b3336d846b089f8df2ba997f15c P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e6e59471bcd4c31b213dfe952991ec4 in term 0.
I20260811 18:41:23.023437 14294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5627b3336d846b089f8df2ba997f15c" candidate_uuid: "3e6e59471bcd4c31b213dfe952991ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" is_pre_election: true
I20260811 18:41:23.024010 14294 raft_consensus.cc:2454] T d5627b3336d846b089f8df2ba997f15c P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e6e59471bcd4c31b213dfe952991ec4 in term 0.
I20260811 18:41:23.024099 13564 leader_election.cc:302] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e6e59471bcd4c31b213dfe952991ec4, 56dd0faaf36046478b6dfddbe800f300; no voters:
I20260811 18:41:23.024588 14439 raft_consensus.cc:2781] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:41:23.024760 14439 raft_consensus.cc:482] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:41:23.024884 14439 raft_consensus.cc:3037] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:23.027171 14439 raft_consensus.cc:504] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } } peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } }
I20260811 18:41:23.027825 14439 leader_election.cc:288] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4 [CANDIDATE]: Term 1 election: Requested vote from peers 56dd0faaf36046478b6dfddbe800f300 (127.30.185.130:40829), c36cf4f91b244ca2b5b6f8f003a5a033 (127.30.185.131:37899)
I20260811 18:41:23.028299 14294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5627b3336d846b089f8df2ba997f15c" candidate_uuid: "3e6e59471bcd4c31b213dfe952991ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033"
I20260811 18:41:23.028302 14022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5627b3336d846b089f8df2ba997f15c" candidate_uuid: "3e6e59471bcd4c31b213dfe952991ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56dd0faaf36046478b6dfddbe800f300"
I20260811 18:41:23.028523 14022 raft_consensus.cc:3037] T d5627b3336d846b089f8df2ba997f15c P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:23.028522 14294 raft_consensus.cc:3037] T d5627b3336d846b089f8df2ba997f15c P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:23.031860 14294 raft_consensus.cc:2454] T d5627b3336d846b089f8df2ba997f15c P c36cf4f91b244ca2b5b6f8f003a5a033 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e6e59471bcd4c31b213dfe952991ec4 in term 1.
I20260811 18:41:23.031860 14022 raft_consensus.cc:2454] T d5627b3336d846b089f8df2ba997f15c P 56dd0faaf36046478b6dfddbe800f300 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e6e59471bcd4c31b213dfe952991ec4 in term 1.
I20260811 18:41:23.032356 13564 leader_election.cc:302] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e6e59471bcd4c31b213dfe952991ec4, 56dd0faaf36046478b6dfddbe800f300; no voters:
I20260811 18:41:23.032665 14439 raft_consensus.cc:2781] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:41:23.033234 14439 raft_consensus.cc:686] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4 [term 1 LEADER]: Becoming Leader. State: Replica: 3e6e59471bcd4c31b213dfe952991ec4, State: Running, Role: LEADER
I20260811 18:41:23.033802 14439 consensus_queue.cc:237] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } } peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } }
I20260811 18:41:23.038554 13381 catalog_manager.cc:5495] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e6e59471bcd4c31b213dfe952991ec4 (127.30.185.129). New cstate: current_term: 1 leader_uuid: "3e6e59471bcd4c31b213dfe952991ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } health_report { overall_health: HEALTHY } } }
I20260811 18:41:23.082250 14428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:23.076575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54420 (local address 127.30.185.129:37959)
0811 18:41:23.076799 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:23.076806 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:23.076827 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:41:23.077070 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:23.077081 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:23.077217 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:23.077383 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:41:23.077767 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.078515 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.079910 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.080345 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.080482 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:23.080977 (+ 495us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:41:23.081590 (+ 613us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:41:23.081753 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:23.081861 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 18:41:23.081958 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 18:41:23.090644 14294 raft_consensus.cc:1261] T d5627b3336d846b089f8df2ba997f15c P c36cf4f91b244ca2b5b6f8f003a5a033 [term 1 FOLLOWER]: Refusing update from remote peer 3e6e59471bcd4c31b213dfe952991ec4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:41:23.090677 14022 raft_consensus.cc:1261] T d5627b3336d846b089f8df2ba997f15c P 56dd0faaf36046478b6dfddbe800f300 [term 1 FOLLOWER]: Refusing update from remote peer 3e6e59471bcd4c31b213dfe952991ec4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:41:23.091707 14448 consensus_queue.cc:1035] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:41:23.092069 14439 consensus_queue.cc:1035] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:41:23.104656 14461 mvcc.cc:205] Tried to move back new op lower bound from 7317396205929009152 to 7317396205710376960. Current Snapshot: MvccSnapshot[applied={T|T < 7317396205929009152}]
I20260811 18:41:23.117728 13380 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:41:23.118079 13380 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:41:23.122769 13380 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877: Sending DeleteTablet for 3 replicas of tablet d5627b3336d846b089f8df2ba997f15c
I20260811 18:41:23.123988 13978 tablet_service.cc:1514] Processing DeleteTablet for tablet d5627b3336d846b089f8df2ba997f15c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:41:23 UTC) from {username='oryx', principal='oryx/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:49974
I20260811 18:41:23.125074 14475 tablet_replica.cc:330] T d5627b3336d846b089f8df2ba997f15c P 56dd0faaf36046478b6dfddbe800f300: stopping tablet replica
I20260811 18:41:23.125764 14252 tablet_service.cc:1514] Processing DeleteTablet for tablet d5627b3336d846b089f8df2ba997f15c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:41:23 UTC) from {username='oryx', principal='oryx/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:52884
I20260811 18:41:23.125777 13675 tablet_service.cc:1514] Processing DeleteTablet for tablet d5627b3336d846b089f8df2ba997f15c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:41:23 UTC) from {username='oryx', principal='oryx/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:54410
I20260811 18:41:23.125855 14475 raft_consensus.cc:2229] T d5627b3336d846b089f8df2ba997f15c P 56dd0faaf36046478b6dfddbe800f300 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:41:23.126510 14475 raft_consensus.cc:2258] T d5627b3336d846b089f8df2ba997f15c P 56dd0faaf36046478b6dfddbe800f300 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:41:23.128772 14475 ts_tablet_manager.cc:1905] T d5627b3336d846b089f8df2ba997f15c P 56dd0faaf36046478b6dfddbe800f300: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:41:23.128789 13381 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:41:23.129520 13381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:41:23.136307 14475 ts_tablet_manager.cc:1918] T d5627b3336d846b089f8df2ba997f15c P 56dd0faaf36046478b6dfddbe800f300: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:41:23.136590 14475 log.cc:1196] T d5627b3336d846b089f8df2ba997f15c P 56dd0faaf36046478b6dfddbe800f300: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/d5627b3336d846b089f8df2ba997f15c
I20260811 18:41:23.137324 14475 ts_tablet_manager.cc:1939] T d5627b3336d846b089f8df2ba997f15c P 56dd0faaf36046478b6dfddbe800f300: Deleting consensus metadata
I20260811 18:41:23.139838 13353 catalog_manager.cc:4841] TS 56dd0faaf36046478b6dfddbe800f300 (127.30.185.130:40829): tablet d5627b3336d846b089f8df2ba997f15c (table test-table [id=e5d8f464574b4d16a88d3bde47283717]) successfully deleted
I20260811 18:41:23.139987 13978 tablet_service.cc:1467] Processing CreateTablet for tablet 1ac1b8bb29e94840a69b1435c45c8dab (DEFAULT_TABLE table=test-table [id=35f7c69b951447829095bbe8965264f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:41:23.140009 14250 tablet_service.cc:1467] Processing CreateTablet for tablet 1ac1b8bb29e94840a69b1435c45c8dab (DEFAULT_TABLE table=test-table [id=35f7c69b951447829095bbe8965264f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:41:23.140419 13978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ac1b8bb29e94840a69b1435c45c8dab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:23.140594 14250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ac1b8bb29e94840a69b1435c45c8dab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:23.146692 13674 tablet_service.cc:1467] Processing CreateTablet for tablet 1ac1b8bb29e94840a69b1435c45c8dab (DEFAULT_TABLE table=test-table [id=35f7c69b951447829095bbe8965264f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:41:23.147338 13674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ac1b8bb29e94840a69b1435c45c8dab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:23.148576 14430 tablet_bootstrap.cc:492] T 1ac1b8bb29e94840a69b1435c45c8dab P 56dd0faaf36046478b6dfddbe800f300: Bootstrap starting.
I20260811 18:41:23.150532 14430 tablet_bootstrap.cc:654] T 1ac1b8bb29e94840a69b1435c45c8dab P 56dd0faaf36046478b6dfddbe800f300: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:23.155776 14479 tablet_replica.cc:330] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4: stopping tablet replica
I20260811 18:41:23.156136 14432 tablet_bootstrap.cc:492] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033: Bootstrap starting.
I20260811 18:41:23.157712 14479 raft_consensus.cc:2229] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:41:23.157959 14432 tablet_bootstrap.cc:654] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:23.158612 14479 raft_consensus.cc:2258] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:41:23.162081 14429 tablet_bootstrap.cc:492] T 1ac1b8bb29e94840a69b1435c45c8dab P 3e6e59471bcd4c31b213dfe952991ec4: Bootstrap starting.
I20260811 18:41:23.163098 14479 ts_tablet_manager.cc:1905] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:41:23.163311 14430 tablet_bootstrap.cc:492] T 1ac1b8bb29e94840a69b1435c45c8dab P 56dd0faaf36046478b6dfddbe800f300: No bootstrap required, opened a new log
I20260811 18:41:23.163514 14430 ts_tablet_manager.cc:1397] T 1ac1b8bb29e94840a69b1435c45c8dab P 56dd0faaf36046478b6dfddbe800f300: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260811 18:41:23.164364 14477 tablet_replica.cc:330] T d5627b3336d846b089f8df2ba997f15c P c36cf4f91b244ca2b5b6f8f003a5a033: stopping tablet replica
I20260811 18:41:23.164547 14430 raft_consensus.cc:348] T 1ac1b8bb29e94840a69b1435c45c8dab P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:23.164798 14430 raft_consensus.cc:374] T 1ac1b8bb29e94840a69b1435c45c8dab P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:23.164902 14477 raft_consensus.cc:2229] T d5627b3336d846b089f8df2ba997f15c P c36cf4f91b244ca2b5b6f8f003a5a033 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:41:23.164991 14430 raft_consensus.cc:729] T 1ac1b8bb29e94840a69b1435c45c8dab P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56dd0faaf36046478b6dfddbe800f300, State: Initialized, Role: FOLLOWER
I20260811 18:41:23.165290 14429 tablet_bootstrap.cc:654] T 1ac1b8bb29e94840a69b1435c45c8dab P 3e6e59471bcd4c31b213dfe952991ec4: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:23.165495 14477 raft_consensus.cc:2258] T d5627b3336d846b089f8df2ba997f15c P c36cf4f91b244ca2b5b6f8f003a5a033 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:41:23.165444 14430 consensus_queue.cc:260] T 1ac1b8bb29e94840a69b1435c45c8dab P 56dd0faaf36046478b6dfddbe800f300 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:23.166476 14432 tablet_bootstrap.cc:492] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033: No bootstrap required, opened a new log
I20260811 18:41:23.166657 14432 ts_tablet_manager.cc:1397] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260811 18:41:23.166846 14430 ts_tablet_manager.cc:1428] T 1ac1b8bb29e94840a69b1435c45c8dab P 56dd0faaf36046478b6dfddbe800f300: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:41:23.167469 14432 raft_consensus.cc:348] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:23.167719 14432 raft_consensus.cc:374] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:23.167827 14432 raft_consensus.cc:729] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36cf4f91b244ca2b5b6f8f003a5a033, State: Initialized, Role: FOLLOWER
I20260811 18:41:23.167851 14477 ts_tablet_manager.cc:1905] T d5627b3336d846b089f8df2ba997f15c P c36cf4f91b244ca2b5b6f8f003a5a033: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:41:23.168311 14432 consensus_queue.cc:260] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:23.169004 14432 ts_tablet_manager.cc:1428] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 18:41:23.173930 14477 ts_tablet_manager.cc:1918] T d5627b3336d846b089f8df2ba997f15c P c36cf4f91b244ca2b5b6f8f003a5a033: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:41:23.174105 14477 log.cc:1196] T d5627b3336d846b089f8df2ba997f15c P c36cf4f91b244ca2b5b6f8f003a5a033: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/d5627b3336d846b089f8df2ba997f15c
I20260811 18:41:23.174691 14477 ts_tablet_manager.cc:1939] T d5627b3336d846b089f8df2ba997f15c P c36cf4f91b244ca2b5b6f8f003a5a033: Deleting consensus metadata
I20260811 18:41:23.176529 13352 catalog_manager.cc:4841] TS c36cf4f91b244ca2b5b6f8f003a5a033 (127.30.185.131:37899): tablet d5627b3336d846b089f8df2ba997f15c (table test-table [id=e5d8f464574b4d16a88d3bde47283717]) successfully deleted
I20260811 18:41:23.178336 14429 tablet_bootstrap.cc:492] T 1ac1b8bb29e94840a69b1435c45c8dab P 3e6e59471bcd4c31b213dfe952991ec4: No bootstrap required, opened a new log
I20260811 18:41:23.178524 14429 ts_tablet_manager.cc:1397] T 1ac1b8bb29e94840a69b1435c45c8dab P 3e6e59471bcd4c31b213dfe952991ec4: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.000s
I20260811 18:41:23.179487 14429 raft_consensus.cc:348] T 1ac1b8bb29e94840a69b1435c45c8dab P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:23.179760 14429 raft_consensus.cc:374] T 1ac1b8bb29e94840a69b1435c45c8dab P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:23.179889 14429 raft_consensus.cc:729] T 1ac1b8bb29e94840a69b1435c45c8dab P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e6e59471bcd4c31b213dfe952991ec4, State: Initialized, Role: FOLLOWER
I20260811 18:41:23.180194 14429 consensus_queue.cc:260] T 1ac1b8bb29e94840a69b1435c45c8dab P 3e6e59471bcd4c31b213dfe952991ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:23.180907 14429 ts_tablet_manager.cc:1428] T 1ac1b8bb29e94840a69b1435c45c8dab P 3e6e59471bcd4c31b213dfe952991ec4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 18:41:23.182169 14479 ts_tablet_manager.cc:1918] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:41:23.182384 14479 log.cc:1196] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-0/wal/wals/d5627b3336d846b089f8df2ba997f15c
I20260811 18:41:23.183072 14479 ts_tablet_manager.cc:1939] T d5627b3336d846b089f8df2ba997f15c P 3e6e59471bcd4c31b213dfe952991ec4: Deleting consensus metadata
I20260811 18:41:23.185236 13355 catalog_manager.cc:4841] TS 3e6e59471bcd4c31b213dfe952991ec4 (127.30.185.129:37959): tablet d5627b3336d846b089f8df2ba997f15c (table test-table [id=e5d8f464574b4d16a88d3bde47283717]) successfully deleted
I20260811 18:41:23.237066 14440 raft_consensus.cc:482] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:41:23.237337 14440 raft_consensus.cc:504] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:23.238884 14440 leader_election.cc:288] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e6e59471bcd4c31b213dfe952991ec4 (127.30.185.129:37959), 56dd0faaf36046478b6dfddbe800f300 (127.30.185.130:40829)
I20260811 18:41:23.247143 14428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:23.239516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:48267 (local address 127.30.185.129:37959)
0811 18:41:23.239714 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:23.239720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:23.240285 (+ 565us) server_negotiation.cc:407] Connection header received
0811 18:41:23.240554 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:23.240563 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:23.240669 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:23.240775 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:23.241394 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.242300 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.244648 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.246009 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.246208 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:23.246286 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:23.246652 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 18:41:23.246798 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 18:41:23.247730 14418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:23.239552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:60873 (local address 127.30.185.130:40829)
0811 18:41:23.239874 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:23.239881 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:23.240606 (+ 725us) server_negotiation.cc:407] Connection header received
0811 18:41:23.240796 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:23.240802 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:23.240878 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:23.240971 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:23.241720 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.242838 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.245054 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.246102 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.246268 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:23.246408 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:23.246954 (+ 546us) server_negotiation.cc:299] Negotiation successful
0811 18:41:23.247082 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193}
I20260811 18:41:23.248198 14486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:23.239112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:37959 (local address 127.30.185.131:48267)
0811 18:41:23.240136 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:23.240167 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:23.240403 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:23.240808 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:23.240812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:23.240837 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:23.241202 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:23.241222 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.242428 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.242435 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:23.244471 (+ 2036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:23.244485 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.246218 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.246225 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:23.246433 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:23.246474 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 18:41:23.247572 (+ 1098us) client_negotiation.cc:240] Negotiation successful
0811 18:41:23.247718 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"thread_start_us":749,"threads_started":1}
I20260811 18:41:23.248282 14487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:23.239117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:40829 (local address 127.30.185.131:60873)
0811 18:41:23.240494 (+ 1377us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:23.240513 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:23.240651 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:23.241004 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:23.241008 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:23.241040 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:23.241543 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:23.241556 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.242990 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.242995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:23.244916 (+ 1921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:23.244926 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.246567 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.246575 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:23.246770 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:23.246801 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 18:41:23.247737 (+ 936us) client_negotiation.cc:240] Negotiation successful
0811 18:41:23.247886 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1228,"mutex_wait_us":53,"thread_start_us":988,"threads_started":1}
I20260811 18:41:23.248970 14022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac1b8bb29e94840a69b1435c45c8dab" candidate_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56dd0faaf36046478b6dfddbe800f300" is_pre_election: true
I20260811 18:41:23.249274 14022 raft_consensus.cc:2454] T 1ac1b8bb29e94840a69b1435c45c8dab P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36cf4f91b244ca2b5b6f8f003a5a033 in term 0.
I20260811 18:41:23.249238 13717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac1b8bb29e94840a69b1435c45c8dab" candidate_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e6e59471bcd4c31b213dfe952991ec4" is_pre_election: true
I20260811 18:41:23.249768 13717 raft_consensus.cc:2454] T 1ac1b8bb29e94840a69b1435c45c8dab P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36cf4f91b244ca2b5b6f8f003a5a033 in term 0.
I20260811 18:41:23.249773 14145 leader_election.cc:302] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56dd0faaf36046478b6dfddbe800f300, c36cf4f91b244ca2b5b6f8f003a5a033; no voters:
I20260811 18:41:23.250185 14440 raft_consensus.cc:2781] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:41:23.250330 14440 raft_consensus.cc:482] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:41:23.250453 14440 raft_consensus.cc:3037] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:23.252642 14440 raft_consensus.cc:504] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:23.253433 14440 leader_election.cc:288] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033 [CANDIDATE]: Term 1 election: Requested vote from peers 3e6e59471bcd4c31b213dfe952991ec4 (127.30.185.129:37959), 56dd0faaf36046478b6dfddbe800f300 (127.30.185.130:40829)
I20260811 18:41:23.253815 13717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac1b8bb29e94840a69b1435c45c8dab" candidate_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e6e59471bcd4c31b213dfe952991ec4"
I20260811 18:41:23.253963 14022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac1b8bb29e94840a69b1435c45c8dab" candidate_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56dd0faaf36046478b6dfddbe800f300"
I20260811 18:41:23.254057 13717 raft_consensus.cc:3037] T 1ac1b8bb29e94840a69b1435c45c8dab P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:23.254200 14022 raft_consensus.cc:3037] T 1ac1b8bb29e94840a69b1435c45c8dab P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:23.257404 14022 raft_consensus.cc:2454] T 1ac1b8bb29e94840a69b1435c45c8dab P 56dd0faaf36046478b6dfddbe800f300 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36cf4f91b244ca2b5b6f8f003a5a033 in term 1.
I20260811 18:41:23.257654 13717 raft_consensus.cc:2454] T 1ac1b8bb29e94840a69b1435c45c8dab P 3e6e59471bcd4c31b213dfe952991ec4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36cf4f91b244ca2b5b6f8f003a5a033 in term 1.
I20260811 18:41:23.257912 14145 leader_election.cc:302] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56dd0faaf36046478b6dfddbe800f300, c36cf4f91b244ca2b5b6f8f003a5a033; no voters:
I20260811 18:41:23.258304 14440 raft_consensus.cc:2781] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:41:23.258827 14440 raft_consensus.cc:686] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033 [term 1 LEADER]: Becoming Leader. State: Replica: c36cf4f91b244ca2b5b6f8f003a5a033, State: Running, Role: LEADER
I20260811 18:41:23.259227 14440 consensus_queue.cc:237] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:23.262979 13381 catalog_manager.cc:5495] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033 reported cstate change: term changed from 0 to 1, leader changed from <none> to c36cf4f91b244ca2b5b6f8f003a5a033 (127.30.185.131). New cstate: current_term: 1 leader_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } health_report { overall_health: HEALTHY } } }
Aug 11 18:41:23 dist-test-slave-z8x9 krb5kdc[13283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473680, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.190@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM
I20260811 18:41:23.315776 14500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:23.299971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:40685 (local address 127.0.0.1:48824)
0811 18:41:23.300680 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:23.300703 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:23.300849 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:23.301271 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:23.301275 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:23.301973 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:23.302229 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:23.302239 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.303701 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.303705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:23.304691 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:23.304703 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.305785 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.305790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:23.305920 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:23.306654 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:23.306671 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:23.309214 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:23.313209 (+ 3995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:23.313232 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:23.313239 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:23.313705 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:23.314126 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:23.314130 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:23.314132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:23.314272 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:23.314927 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:23.314934 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:23.315090 (+ 156us) client_negotiation.cc:769] Sending connection context
0811 18:41:23.315219 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 18:41:23.315438 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":409,"threads_started":1}
I20260811 18:41:23.316275 13873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:23.299958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48824 (local address 127.30.185.190:40685)
0811 18:41:23.300224 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:23.300231 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:23.300800 (+ 569us) server_negotiation.cc:407] Connection header received
0811 18:41:23.300973 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:23.300979 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:23.301063 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:23.301223 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:23.302396 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.303563 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.304870 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.305608 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.305741 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:23.309425 (+ 3684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:23.309433 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:23.309439 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:23.309479 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:23.313011 (+ 3532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:23.313824 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:23.313838 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:23.313842 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:23.314012 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:23.314383 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:23.314388 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:23.314391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:23.314775 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:23.314938 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:23.315370 (+ 432us) server_negotiation.cc:299] Negotiation successful
0811 18:41:23.315615 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 18:41:23.321587 13381 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:48824:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 18:41:23.322810 13381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:41:23.330835 13978 tablet_service.cc:1467] Processing CreateTablet for tablet a6c983bb30c541c6b7fe787962127a99 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e6a1daba9826460bba485f0f704e9441]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:41:23.330888 13674 tablet_service.cc:1467] Processing CreateTablet for tablet a6c983bb30c541c6b7fe787962127a99 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e6a1daba9826460bba485f0f704e9441]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:41:23.331135 14252 tablet_service.cc:1467] Processing CreateTablet for tablet a6c983bb30c541c6b7fe787962127a99 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e6a1daba9826460bba485f0f704e9441]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:41:23.331452 13674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6c983bb30c541c6b7fe787962127a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:23.331457 13978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6c983bb30c541c6b7fe787962127a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:23.331684 14252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6c983bb30c541c6b7fe787962127a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:23.339319 14429 tablet_bootstrap.cc:492] T a6c983bb30c541c6b7fe787962127a99 P 3e6e59471bcd4c31b213dfe952991ec4: Bootstrap starting.
I20260811 18:41:23.339313 14430 tablet_bootstrap.cc:492] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300: Bootstrap starting.
I20260811 18:41:23.340114 14432 tablet_bootstrap.cc:492] T a6c983bb30c541c6b7fe787962127a99 P c36cf4f91b244ca2b5b6f8f003a5a033: Bootstrap starting.
I20260811 18:41:23.341199 14429 tablet_bootstrap.cc:654] T a6c983bb30c541c6b7fe787962127a99 P 3e6e59471bcd4c31b213dfe952991ec4: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:23.341199 14430 tablet_bootstrap.cc:654] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:23.341820 14432 tablet_bootstrap.cc:654] T a6c983bb30c541c6b7fe787962127a99 P c36cf4f91b244ca2b5b6f8f003a5a033: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:23.352051 14432 tablet_bootstrap.cc:492] T a6c983bb30c541c6b7fe787962127a99 P c36cf4f91b244ca2b5b6f8f003a5a033: No bootstrap required, opened a new log
I20260811 18:41:23.352241 14432 ts_tablet_manager.cc:1397] T a6c983bb30c541c6b7fe787962127a99 P c36cf4f91b244ca2b5b6f8f003a5a033: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260811 18:41:23.352916 14429 tablet_bootstrap.cc:492] T a6c983bb30c541c6b7fe787962127a99 P 3e6e59471bcd4c31b213dfe952991ec4: No bootstrap required, opened a new log
I20260811 18:41:23.352914 14432 raft_consensus.cc:348] T a6c983bb30c541c6b7fe787962127a99 P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:23.353096 14429 ts_tablet_manager.cc:1397] T a6c983bb30c541c6b7fe787962127a99 P 3e6e59471bcd4c31b213dfe952991ec4: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.005s
I20260811 18:41:23.353770 14430 tablet_bootstrap.cc:492] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300: No bootstrap required, opened a new log
I20260811 18:41:23.353961 14430 ts_tablet_manager.cc:1397] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.000s
I20260811 18:41:23.353844 14429 raft_consensus.cc:348] T a6c983bb30c541c6b7fe787962127a99 P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:23.354086 14429 raft_consensus.cc:374] T a6c983bb30c541c6b7fe787962127a99 P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:23.354168 14429 raft_consensus.cc:729] T a6c983bb30c541c6b7fe787962127a99 P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e6e59471bcd4c31b213dfe952991ec4, State: Initialized, Role: FOLLOWER
I20260811 18:41:23.354466 14429 consensus_queue.cc:260] T a6c983bb30c541c6b7fe787962127a99 P 3e6e59471bcd4c31b213dfe952991ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:23.354650 14430 raft_consensus.cc:348] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:23.354918 14430 raft_consensus.cc:374] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:23.355000 14430 raft_consensus.cc:729] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56dd0faaf36046478b6dfddbe800f300, State: Initialized, Role: FOLLOWER
I20260811 18:41:23.354914 14448 tablet_replica.cc:439] T a6c983bb30c541c6b7fe787962127a99 P 3e6e59471bcd4c31b213dfe952991ec4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } } }
I20260811 18:41:23.355238 14448 tablet_replica.cc:442] T a6c983bb30c541c6b7fe787962127a99 P 3e6e59471bcd4c31b213dfe952991ec4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:41:23.355234 14430 consensus_queue.cc:260] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:23.355561 14432 raft_consensus.cc:374] T a6c983bb30c541c6b7fe787962127a99 P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:23.355674 14432 raft_consensus.cc:729] T a6c983bb30c541c6b7fe787962127a99 P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36cf4f91b244ca2b5b6f8f003a5a033, State: Initialized, Role: FOLLOWER
I20260811 18:41:23.355856 14429 ts_tablet_manager.cc:1428] T a6c983bb30c541c6b7fe787962127a99 P 3e6e59471bcd4c31b213dfe952991ec4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:41:23.355947 14432 consensus_queue.cc:260] T a6c983bb30c541c6b7fe787962127a99 P c36cf4f91b244ca2b5b6f8f003a5a033 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:23.356395 14440 tablet_replica.cc:439] T a6c983bb30c541c6b7fe787962127a99 P c36cf4f91b244ca2b5b6f8f003a5a033: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } } }
I20260811 18:41:23.356707 14440 tablet_replica.cc:442] T a6c983bb30c541c6b7fe787962127a99 P c36cf4f91b244ca2b5b6f8f003a5a033: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:41:23.357569 14432 ts_tablet_manager.cc:1428] T a6c983bb30c541c6b7fe787962127a99 P c36cf4f91b244ca2b5b6f8f003a5a033: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260811 18:41:23.358119 14430 ts_tablet_manager.cc:1428] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260811 18:41:23.357986 14441 tablet_replica.cc:439] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } } }
I20260811 18:41:23.358407 14441 tablet_replica.cc:442] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300: This TxnStatusTablet replica's current role is: FOLLOWER
W20260811 18:41:23.405391 13819 tablet.cc:2369] T 1ac1b8bb29e94840a69b1435c45c8dab P 3e6e59471bcd4c31b213dfe952991ec4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:41:23.616554 14441 raft_consensus.cc:482] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:41:23.616845 14441 raft_consensus.cc:504] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:23.618724 14441 leader_election.cc:288] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e6e59471bcd4c31b213dfe952991ec4 (127.30.185.129:37959), c36cf4f91b244ca2b5b6f8f003a5a033 (127.30.185.131:37899)
I20260811 18:41:23.627842 14428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:23.619073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:35187 (local address 127.30.185.129:37959)
0811 18:41:23.619280 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:23.619287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:23.620072 (+ 785us) server_negotiation.cc:407] Connection header received
0811 18:41:23.620314 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:23.620318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:23.620382 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:23.620499 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:23.621236 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.622330 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.625315 (+ 2985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.626561 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.626759 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:23.626837 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:23.627373 (+ 536us) server_negotiation.cc:299] Negotiation successful
0811 18:41:23.627529 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 18:41:23.627862 14511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:23.619347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:47279 (local address 127.30.185.131:37899)
0811 18:41:23.619848 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:23.619856 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:23.620455 (+ 599us) server_negotiation.cc:407] Connection header received
0811 18:41:23.620721 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:23.620726 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:23.620792 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:23.620869 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:23.621673 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.622658 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.625423 (+ 2765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.626641 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.626794 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:23.626864 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:23.627369 (+ 505us) server_negotiation.cc:299] Negotiation successful
0811 18:41:23.627530 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":302,"threads_started":1}
I20260811 18:41:23.628090 14512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:23.619079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:37899 (local address 127.30.185.130:47279)
0811 18:41:23.620357 (+ 1278us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:23.620373 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:23.620544 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:23.620951 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:23.620956 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:23.620988 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:23.621407 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:23.621416 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.622785 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.622790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:23.625254 (+ 2464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:23.625269 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.626872 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.626878 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:23.627137 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:23.627186 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 18:41:23.627303 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 18:41:23.627553 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1151,"mutex_wait_us":126,"thread_start_us":936,"threads_started":1}
I20260811 18:41:23.628481 14510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:23.618932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:37959 (local address 127.30.185.130:35187)
0811 18:41:23.619925 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:23.619955 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:23.620152 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:23.620539 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:23.620545 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:23.620569 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:23.620986 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:23.620998 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.622463 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.622467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:23.625101 (+ 2634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:23.625118 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.626828 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.626838 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:23.627121 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:23.627199 (+ 78us) client_negotiation.cc:769] Sending connection context
0811 18:41:23.627299 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 18:41:23.627553 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":854,"mutex_wait_us":165,"thread_start_us":737,"threads_started":1}
I20260811 18:41:23.628827 14294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6c983bb30c541c6b7fe787962127a99" candidate_uuid: "56dd0faaf36046478b6dfddbe800f300" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" is_pre_election: true
I20260811 18:41:23.629117 14294 raft_consensus.cc:2454] T a6c983bb30c541c6b7fe787962127a99 P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56dd0faaf36046478b6dfddbe800f300 in term 0.
I20260811 18:41:23.629311 13717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6c983bb30c541c6b7fe787962127a99" candidate_uuid: "56dd0faaf36046478b6dfddbe800f300" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e6e59471bcd4c31b213dfe952991ec4" is_pre_election: true
I20260811 18:41:23.629603 13717 raft_consensus.cc:2454] T a6c983bb30c541c6b7fe787962127a99 P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56dd0faaf36046478b6dfddbe800f300 in term 0.
I20260811 18:41:23.629688 13859 leader_election.cc:302] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56dd0faaf36046478b6dfddbe800f300, c36cf4f91b244ca2b5b6f8f003a5a033; no voters:
I20260811 18:41:23.630365 14441 raft_consensus.cc:2781] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:41:23.630465 14441 raft_consensus.cc:482] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:41:23.630529 14441 raft_consensus.cc:3037] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:23.632730 14441 raft_consensus.cc:504] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:23.633560 14441 leader_election.cc:288] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300 [CANDIDATE]: Term 1 election: Requested vote from peers 3e6e59471bcd4c31b213dfe952991ec4 (127.30.185.129:37959), c36cf4f91b244ca2b5b6f8f003a5a033 (127.30.185.131:37899)
I20260811 18:41:23.634045 13717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6c983bb30c541c6b7fe787962127a99" candidate_uuid: "56dd0faaf36046478b6dfddbe800f300" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e6e59471bcd4c31b213dfe952991ec4"
I20260811 18:41:23.634210 14294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6c983bb30c541c6b7fe787962127a99" candidate_uuid: "56dd0faaf36046478b6dfddbe800f300" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033"
I20260811 18:41:23.634305 13717 raft_consensus.cc:3037] T a6c983bb30c541c6b7fe787962127a99 P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:23.634488 14294 raft_consensus.cc:3037] T a6c983bb30c541c6b7fe787962127a99 P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:23.636791 13717 raft_consensus.cc:2454] T a6c983bb30c541c6b7fe787962127a99 P 3e6e59471bcd4c31b213dfe952991ec4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56dd0faaf36046478b6dfddbe800f300 in term 1.
I20260811 18:41:23.636843 14294 raft_consensus.cc:2454] T a6c983bb30c541c6b7fe787962127a99 P c36cf4f91b244ca2b5b6f8f003a5a033 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56dd0faaf36046478b6dfddbe800f300 in term 1.
I20260811 18:41:23.637328 13863 leader_election.cc:302] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e6e59471bcd4c31b213dfe952991ec4, 56dd0faaf36046478b6dfddbe800f300; no voters:
I20260811 18:41:23.637713 14441 raft_consensus.cc:2781] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:41:23.638116 14441 raft_consensus.cc:686] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300 [term 1 LEADER]: Becoming Leader. State: Replica: 56dd0faaf36046478b6dfddbe800f300, State: Running, Role: LEADER
I20260811 18:41:23.638520 14441 consensus_queue.cc:237] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:23.640502 14513 tablet_replica.cc:439] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300: TxnStatusTablet state changed. Reason: New leader 56dd0faaf36046478b6dfddbe800f300. Latest consensus state: current_term: 1 leader_uuid: "56dd0faaf36046478b6dfddbe800f300" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } } }
I20260811 18:41:23.640906 14513 tablet_replica.cc:442] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:41:23.642590 13381 catalog_manager.cc:5495] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56dd0faaf36046478b6dfddbe800f300 (127.30.185.130). New cstate: current_term: 1 leader_uuid: "56dd0faaf36046478b6dfddbe800f300" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } health_report { overall_health: UNKNOWN } } }
I20260811 18:41:23.645716 14518 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 18:41:23.645982 14518 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 18:41:23.730278 14418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:23.723202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49980 (local address 127.30.185.130:40829)
0811 18:41:23.723379 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:23.723384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:23.723590 (+ 206us) server_negotiation.cc:407] Connection header received
0811 18:41:23.723946 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:23.723951 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:23.724047 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:23.724281 (+ 234us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:41:23.724718 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.725704 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.727848 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.728320 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.728428 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:23.728998 (+ 570us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:41:23.729654 (+ 656us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:41:23.729766 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:23.729936 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 18:41:23.730014 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 18:41:23.730531 14500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:23.723082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:40829 (local address 127.0.0.1:49980)
0811 18:41:23.723494 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:23.723517 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:23.723684 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:23.724310 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:23.724317 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:23.724334 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:41:23.724566 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:23.724578 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.725838 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.725845 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:23.727679 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:23.727696 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.728487 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.728494 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:23.728726 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:23.728810 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:41:23.729771 (+ 961us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:41:23.729800 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 18:41:23.729911 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 18:41:23.730107 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224}
I20260811 18:41:23.735631 14294 raft_consensus.cc:1261] T a6c983bb30c541c6b7fe787962127a99 P c36cf4f91b244ca2b5b6f8f003a5a033 [term 1 FOLLOWER]: Refusing update from remote peer 56dd0faaf36046478b6dfddbe800f300: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:41:23.735788 13717 raft_consensus.cc:1261] T a6c983bb30c541c6b7fe787962127a99 P 3e6e59471bcd4c31b213dfe952991ec4 [term 1 FOLLOWER]: Refusing update from remote peer 56dd0faaf36046478b6dfddbe800f300: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:41:23.736527 14513 consensus_queue.cc:1035] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:41:23.736886 14441 consensus_queue.cc:1035] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:41:23.742136 14448 tablet_replica.cc:439] T a6c983bb30c541c6b7fe787962127a99 P 3e6e59471bcd4c31b213dfe952991ec4: TxnStatusTablet state changed. Reason: New leader 56dd0faaf36046478b6dfddbe800f300. Latest consensus state: current_term: 1 leader_uuid: "56dd0faaf36046478b6dfddbe800f300" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } } }
I20260811 18:41:23.742496 14448 tablet_replica.cc:442] T a6c983bb30c541c6b7fe787962127a99 P 3e6e59471bcd4c31b213dfe952991ec4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:41:23.745854 14513 tablet_replica.cc:439] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "56dd0faaf36046478b6dfddbe800f300" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } } }
I20260811 18:41:23.746212 14528 mvcc.cc:205] Tried to move back new op lower bound from 7317396208572989440 to 7317396208187609088. Current Snapshot: MvccSnapshot[applied={T|T < 7317396208572989440}]
I20260811 18:41:23.746302 14513 tablet_replica.cc:442] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:41:23.747772 14448 tablet_replica.cc:439] T a6c983bb30c541c6b7fe787962127a99 P 3e6e59471bcd4c31b213dfe952991ec4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "56dd0faaf36046478b6dfddbe800f300" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } } }
I20260811 18:41:23.748055 14448 tablet_replica.cc:442] T a6c983bb30c541c6b7fe787962127a99 P 3e6e59471bcd4c31b213dfe952991ec4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:41:23.755038 14440 tablet_replica.cc:439] T a6c983bb30c541c6b7fe787962127a99 P c36cf4f91b244ca2b5b6f8f003a5a033: TxnStatusTablet state changed. Reason: New leader 56dd0faaf36046478b6dfddbe800f300. Latest consensus state: current_term: 1 leader_uuid: "56dd0faaf36046478b6dfddbe800f300" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } } }
I20260811 18:41:23.755327 14440 tablet_replica.cc:442] T a6c983bb30c541c6b7fe787962127a99 P c36cf4f91b244ca2b5b6f8f003a5a033: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:41:23.758103 14513 tablet_replica.cc:439] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "56dd0faaf36046478b6dfddbe800f300" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } } }
I20260811 18:41:23.758395 14513 tablet_replica.cc:442] T a6c983bb30c541c6b7fe787962127a99 P 56dd0faaf36046478b6dfddbe800f300: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:41:23.765465 14440 tablet_replica.cc:439] T a6c983bb30c541c6b7fe787962127a99 P c36cf4f91b244ca2b5b6f8f003a5a033: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "56dd0faaf36046478b6dfddbe800f300" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } } }
I20260811 18:41:23.765731 14440 tablet_replica.cc:442] T a6c983bb30c541c6b7fe787962127a99 P c36cf4f91b244ca2b5b6f8f003a5a033: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:41:23.778805 14511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:23.771379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52896 (local address 127.30.185.131:37899)
0811 18:41:23.771550 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:23.771558 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:23.771879 (+ 321us) server_negotiation.cc:407] Connection header received
0811 18:41:23.772119 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:23.772125 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:23.772248 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:23.772394 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:41:23.772919 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.774093 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.776110 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.776880 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.776996 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:23.777516 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:41:23.778182 (+ 666us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:41:23.778336 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:23.778426 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 18:41:23.778521 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:41:23.797465 14418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:23.790412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:49229 (local address 127.30.185.130:40829)
0811 18:41:23.790641 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:23.790647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:23.790744 (+ 97us) server_negotiation.cc:407] Connection header received
0811 18:41:23.791075 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:23.791080 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:23.791197 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:23.791311 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:41:23.791855 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.793028 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.795212 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.795946 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.796070 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:23.796601 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:41:23.796756 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:41:23.796842 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:23.796998 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 18:41:23.797092 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 18:41:23.797827 14544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:23.789986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:40829 (local address 127.30.185.131:49229)
0811 18:41:23.790625 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:23.790647 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:23.790907 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:23.791328 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:23.791333 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:23.791347 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:41:23.791667 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:23.791684 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.793241 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.793247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:23.795052 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:23.795069 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.796130 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.796136 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:23.796316 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:23.796392 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:41:23.796859 (+ 467us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:41:23.796872 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 18:41:23.796958 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 18:41:23.797138 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":694,"threads_started":1}
I20260811 18:41:23.808761 14022 raft_consensus.cc:1261] T 1ac1b8bb29e94840a69b1435c45c8dab P 56dd0faaf36046478b6dfddbe800f300 [term 1 FOLLOWER]: Refusing update from remote peer c36cf4f91b244ca2b5b6f8f003a5a033: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:41:23.808969 13716 raft_consensus.cc:1261] T 1ac1b8bb29e94840a69b1435c45c8dab P 3e6e59471bcd4c31b213dfe952991ec4 [term 1 FOLLOWER]: Refusing update from remote peer c36cf4f91b244ca2b5b6f8f003a5a033: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:41:23.809604 14545 consensus_queue.cc:1035] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:41:23.810067 14440 consensus_queue.cc:1035] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:41:23.815892 14523 mvcc.cc:205] Tried to move back new op lower bound from 7317396208874737664 to 7317396206633656320. Current Snapshot: MvccSnapshot[applied={T|T < 7317396208874737664}]
I20260811 18:41:23.855474 14511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:23.848260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:55929 (local address 127.30.185.131:37899)
0811 18:41:23.848454 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:23.848461 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:23.849379 (+ 918us) server_negotiation.cc:407] Connection header received
0811 18:41:23.849615 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:23.849622 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:23.849717 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:23.849814 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:41:23.850469 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.851503 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.853273 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.853832 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.853955 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:23.854614 (+ 659us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:41:23.854767 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:41:23.854834 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:23.855108 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 18:41:23.855197 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 18:41:23.856187 14555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:23.848377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:37899 (local address 127.30.185.130:55929)
0811 18:41:23.849260 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:23.849286 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:23.849475 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:23.849853 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:23.849858 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:23.849879 (+ 21us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:41:23.850290 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:23.850302 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.851639 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.851645 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:23.853067 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:23.853084 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:23.854156 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:23.854164 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:23.854314 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:23.854390 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:41:23.854931 (+ 541us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:41:23.854941 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 18:41:23.855688 (+ 747us) client_negotiation.cc:240] Negotiation successful
0811 18:41:23.855839 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":468,"threads_started":1}
I20260811 18:41:23.895649 13381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:41:23.895934 13381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:41:23.899758 13381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877: Sending DeleteTablet for 3 replicas of tablet 1ac1b8bb29e94840a69b1435c45c8dab
I20260811 18:41:23.901081 13674 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ac1b8bb29e94840a69b1435c45c8dab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:41:23 UTC) from {username='oryx', principal='oryx/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:54410
I20260811 18:41:23.901611 13978 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ac1b8bb29e94840a69b1435c45c8dab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:41:23 UTC) from {username='oryx', principal='oryx/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:49974
I20260811 18:41:23.901783 14252 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ac1b8bb29e94840a69b1435c45c8dab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:41:23 UTC) from {username='oryx', principal='oryx/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:52884
I20260811 18:41:23.919723 14563 tablet_replica.cc:330] T 1ac1b8bb29e94840a69b1435c45c8dab P 3e6e59471bcd4c31b213dfe952991ec4: stopping tablet replica
I20260811 18:41:23.920050 14563 raft_consensus.cc:2229] T 1ac1b8bb29e94840a69b1435c45c8dab P 3e6e59471bcd4c31b213dfe952991ec4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:41:23.920362 14563 raft_consensus.cc:2258] T 1ac1b8bb29e94840a69b1435c45c8dab P 3e6e59471bcd4c31b213dfe952991ec4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:41:23.930771 14568 tablet_replica.cc:330] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033: stopping tablet replica
I20260811 18:41:23.931051 14568 raft_consensus.cc:2229] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:41:23.931790 14568 raft_consensus.cc:2258] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:41:23.933669 14568 ts_tablet_manager.cc:1905] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:41:23.934080 14563 ts_tablet_manager.cc:1905] T 1ac1b8bb29e94840a69b1435c45c8dab P 3e6e59471bcd4c31b213dfe952991ec4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:41:23.936790 14565 tablet_replica.cc:330] T 1ac1b8bb29e94840a69b1435c45c8dab P 56dd0faaf36046478b6dfddbe800f300: stopping tablet replica
I20260811 18:41:23.937043 14565 raft_consensus.cc:2229] T 1ac1b8bb29e94840a69b1435c45c8dab P 56dd0faaf36046478b6dfddbe800f300 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:41:23.937386 14565 raft_consensus.cc:2258] T 1ac1b8bb29e94840a69b1435c45c8dab P 56dd0faaf36046478b6dfddbe800f300 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:41:23.941814 14563 ts_tablet_manager.cc:1918] T 1ac1b8bb29e94840a69b1435c45c8dab P 3e6e59471bcd4c31b213dfe952991ec4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:41:23.942013 14563 log.cc:1196] T 1ac1b8bb29e94840a69b1435c45c8dab P 3e6e59471bcd4c31b213dfe952991ec4: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-0/wal/wals/1ac1b8bb29e94840a69b1435c45c8dab
I20260811 18:41:23.942538 14563 ts_tablet_manager.cc:1939] T 1ac1b8bb29e94840a69b1435c45c8dab P 3e6e59471bcd4c31b213dfe952991ec4: Deleting consensus metadata
I20260811 18:41:23.943447 14568 ts_tablet_manager.cc:1918] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:41:23.943626 14568 log.cc:1196] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/1ac1b8bb29e94840a69b1435c45c8dab
I20260811 18:41:23.943991 13355 catalog_manager.cc:4841] TS 3e6e59471bcd4c31b213dfe952991ec4 (127.30.185.129:37959): tablet 1ac1b8bb29e94840a69b1435c45c8dab (table test-table [id=35f7c69b951447829095bbe8965264f1]) successfully deleted
I20260811 18:41:23.944195 14568 ts_tablet_manager.cc:1939] T 1ac1b8bb29e94840a69b1435c45c8dab P c36cf4f91b244ca2b5b6f8f003a5a033: Deleting consensus metadata
I20260811 18:41:23.946009 13352 catalog_manager.cc:4841] TS c36cf4f91b244ca2b5b6f8f003a5a033 (127.30.185.131:37899): tablet 1ac1b8bb29e94840a69b1435c45c8dab (table test-table [id=35f7c69b951447829095bbe8965264f1]) successfully deleted
I20260811 18:41:23.947145 14565 ts_tablet_manager.cc:1905] T 1ac1b8bb29e94840a69b1435c45c8dab P 56dd0faaf36046478b6dfddbe800f300: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:41:23.956897 14565 ts_tablet_manager.cc:1918] T 1ac1b8bb29e94840a69b1435c45c8dab P 56dd0faaf36046478b6dfddbe800f300: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:41:23.957090 14565 log.cc:1196] T 1ac1b8bb29e94840a69b1435c45c8dab P 56dd0faaf36046478b6dfddbe800f300: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/1ac1b8bb29e94840a69b1435c45c8dab
I20260811 18:41:23.957681 14565 ts_tablet_manager.cc:1939] T 1ac1b8bb29e94840a69b1435c45c8dab P 56dd0faaf36046478b6dfddbe800f300: Deleting consensus metadata
I20260811 18:41:23.959151 13353 catalog_manager.cc:4841] TS 56dd0faaf36046478b6dfddbe800f300 (127.30.185.130:40829): tablet 1ac1b8bb29e94840a69b1435c45c8dab (table test-table [id=35f7c69b951447829095bbe8965264f1]) successfully deleted
Aug 11 18:41:23 dist-test-slave-z8x9 krb5kdc[13283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:41:24 dist-test-slave-z8x9 krb5kdc[13283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:24 dist-test-slave-z8x9 krb5kdc[13283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 18:41:24.052248 14581 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:24.027990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48830 (local address 127.30.185.190:40685)
0811 18:41:24.042217 (+ 14227us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:24.042222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:24.042242 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:41:24.042328 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:24.042332 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:24.042394 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:24.042546 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:24.044122 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:24.044910 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:24.046252 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:24.046680 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:24.046817 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:24.051949 (+ 5132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48830: BlockingRecv error: recv got EOF from 127.0.0.1:48830 (error 108)
Metrics: {"server-negotiator.queue_time_us":14118,"thread_start_us":13958,"threads_started":1}
Aug 11 18:41:24 dist-test-slave-z8x9 krb5kdc[13283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM
I20260811 18:41:24.071780 14581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:24.056436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48842 (local address 127.30.185.190:40685)
0811 18:41:24.056677 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:24.056686 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:24.057366 (+ 680us) server_negotiation.cc:407] Connection header received
0811 18:41:24.057576 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:24.057581 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:24.057650 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:24.057754 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:24.059021 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:24.059771 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:24.061009 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:24.061496 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:24.061628 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:24.065805 (+ 4177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:24.065811 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:24.065814 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:24.065847 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:24.068902 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:24.069618 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:24.069625 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:24.069629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:24.069773 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:24.070174 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:24.070179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:24.070186 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:24.070476 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:24.070734 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:24.071101 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 18:41:24.071409 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 18:41:24.075331 13381 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:41:24.076021 13381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:41:24.082718 13674 tablet_service.cc:1467] Processing CreateTablet for tablet d503c1292f534b7295ce1c07ea4b5dd0 (DEFAULT_TABLE table=test-table [id=e743783c988b4550be1ae1f22ee28e87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:41:24.083278 13674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d503c1292f534b7295ce1c07ea4b5dd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:24.083246 13978 tablet_service.cc:1467] Processing CreateTablet for tablet d503c1292f534b7295ce1c07ea4b5dd0 (DEFAULT_TABLE table=test-table [id=e743783c988b4550be1ae1f22ee28e87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:41:24.083503 14252 tablet_service.cc:1467] Processing CreateTablet for tablet d503c1292f534b7295ce1c07ea4b5dd0 (DEFAULT_TABLE table=test-table [id=e743783c988b4550be1ae1f22ee28e87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:41:24.083680 13978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d503c1292f534b7295ce1c07ea4b5dd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:24.083959 14252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d503c1292f534b7295ce1c07ea4b5dd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:24.096100 14601 tablet_bootstrap.cc:492] T d503c1292f534b7295ce1c07ea4b5dd0 P c36cf4f91b244ca2b5b6f8f003a5a033: Bootstrap starting.
I20260811 18:41:24.098649 14601 tablet_bootstrap.cc:654] T d503c1292f534b7295ce1c07ea4b5dd0 P c36cf4f91b244ca2b5b6f8f003a5a033: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:24.099792 14598 tablet_bootstrap.cc:492] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4: Bootstrap starting.
I20260811 18:41:24.100847 14599 tablet_bootstrap.cc:492] T d503c1292f534b7295ce1c07ea4b5dd0 P 56dd0faaf36046478b6dfddbe800f300: Bootstrap starting.
I20260811 18:41:24.102906 14598 tablet_bootstrap.cc:654] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:24.102936 14599 tablet_bootstrap.cc:654] T d503c1292f534b7295ce1c07ea4b5dd0 P 56dd0faaf36046478b6dfddbe800f300: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:24.106638 14601 tablet_bootstrap.cc:492] T d503c1292f534b7295ce1c07ea4b5dd0 P c36cf4f91b244ca2b5b6f8f003a5a033: No bootstrap required, opened a new log
I20260811 18:41:24.106868 14601 ts_tablet_manager.cc:1397] T d503c1292f534b7295ce1c07ea4b5dd0 P c36cf4f91b244ca2b5b6f8f003a5a033: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260811 18:41:24.108057 14601 raft_consensus.cc:348] T d503c1292f534b7295ce1c07ea4b5dd0 P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:24.108433 14601 raft_consensus.cc:374] T d503c1292f534b7295ce1c07ea4b5dd0 P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:24.108532 14601 raft_consensus.cc:729] T d503c1292f534b7295ce1c07ea4b5dd0 P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36cf4f91b244ca2b5b6f8f003a5a033, State: Initialized, Role: FOLLOWER
I20260811 18:41:24.109576 14601 consensus_queue.cc:260] T d503c1292f534b7295ce1c07ea4b5dd0 P c36cf4f91b244ca2b5b6f8f003a5a033 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:24.110262 14601 ts_tablet_manager.cc:1428] T d503c1292f534b7295ce1c07ea4b5dd0 P c36cf4f91b244ca2b5b6f8f003a5a033: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 18:41:24.113539 14599 tablet_bootstrap.cc:492] T d503c1292f534b7295ce1c07ea4b5dd0 P 56dd0faaf36046478b6dfddbe800f300: No bootstrap required, opened a new log
I20260811 18:41:24.113749 14599 ts_tablet_manager.cc:1397] T d503c1292f534b7295ce1c07ea4b5dd0 P 56dd0faaf36046478b6dfddbe800f300: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.005s
I20260811 18:41:24.114845 14599 raft_consensus.cc:348] T d503c1292f534b7295ce1c07ea4b5dd0 P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:24.115142 14599 raft_consensus.cc:374] T d503c1292f534b7295ce1c07ea4b5dd0 P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:24.115257 14599 raft_consensus.cc:729] T d503c1292f534b7295ce1c07ea4b5dd0 P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56dd0faaf36046478b6dfddbe800f300, State: Initialized, Role: FOLLOWER
I20260811 18:41:24.115302 14598 tablet_bootstrap.cc:492] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4: No bootstrap required, opened a new log
I20260811 18:41:24.115566 14598 ts_tablet_manager.cc:1397] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.005s
I20260811 18:41:24.115572 14599 consensus_queue.cc:260] T d503c1292f534b7295ce1c07ea4b5dd0 P 56dd0faaf36046478b6dfddbe800f300 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:24.117930 14598 raft_consensus.cc:348] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:24.118278 14598 raft_consensus.cc:374] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:24.118453 14598 raft_consensus.cc:729] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e6e59471bcd4c31b213dfe952991ec4, State: Initialized, Role: FOLLOWER
I20260811 18:41:24.118804 14598 consensus_queue.cc:260] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:24.119017 14599 ts_tablet_manager.cc:1428] T d503c1292f534b7295ce1c07ea4b5dd0 P 56dd0faaf36046478b6dfddbe800f300: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260811 18:41:24.120168 14598 ts_tablet_manager.cc:1428] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260811 18:41:24.246223 14448 raft_consensus.cc:482] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:41:24.246491 14448 raft_consensus.cc:504] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:24.247295 14448 leader_election.cc:288] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56dd0faaf36046478b6dfddbe800f300 (127.30.185.130:40829), c36cf4f91b244ca2b5b6f8f003a5a033 (127.30.185.131:37899)
I20260811 18:41:24.247964 14022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d503c1292f534b7295ce1c07ea4b5dd0" candidate_uuid: "3e6e59471bcd4c31b213dfe952991ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56dd0faaf36046478b6dfddbe800f300" is_pre_election: true
I20260811 18:41:24.247992 14294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d503c1292f534b7295ce1c07ea4b5dd0" candidate_uuid: "3e6e59471bcd4c31b213dfe952991ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" is_pre_election: true
I20260811 18:41:24.248224 14022 raft_consensus.cc:2454] T d503c1292f534b7295ce1c07ea4b5dd0 P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e6e59471bcd4c31b213dfe952991ec4 in term 0.
I20260811 18:41:24.248296 14294 raft_consensus.cc:2454] T d503c1292f534b7295ce1c07ea4b5dd0 P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e6e59471bcd4c31b213dfe952991ec4 in term 0.
I20260811 18:41:24.248910 13558 leader_election.cc:302] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3e6e59471bcd4c31b213dfe952991ec4, 56dd0faaf36046478b6dfddbe800f300, c36cf4f91b244ca2b5b6f8f003a5a033; no voters:
I20260811 18:41:24.249281 14448 raft_consensus.cc:2781] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:41:24.249411 14448 raft_consensus.cc:482] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:41:24.249531 14448 raft_consensus.cc:3037] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:24.251662 14448 raft_consensus.cc:504] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:24.252292 14448 leader_election.cc:288] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4 [CANDIDATE]: Term 1 election: Requested vote from peers 56dd0faaf36046478b6dfddbe800f300 (127.30.185.130:40829), c36cf4f91b244ca2b5b6f8f003a5a033 (127.30.185.131:37899)
I20260811 18:41:24.252841 14294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d503c1292f534b7295ce1c07ea4b5dd0" candidate_uuid: "3e6e59471bcd4c31b213dfe952991ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033"
I20260811 18:41:24.252825 14022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d503c1292f534b7295ce1c07ea4b5dd0" candidate_uuid: "3e6e59471bcd4c31b213dfe952991ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56dd0faaf36046478b6dfddbe800f300"
I20260811 18:41:24.253062 14294 raft_consensus.cc:3037] T d503c1292f534b7295ce1c07ea4b5dd0 P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:24.253080 14022 raft_consensus.cc:3037] T d503c1292f534b7295ce1c07ea4b5dd0 P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:24.255419 14294 raft_consensus.cc:2454] T d503c1292f534b7295ce1c07ea4b5dd0 P c36cf4f91b244ca2b5b6f8f003a5a033 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e6e59471bcd4c31b213dfe952991ec4 in term 1.
I20260811 18:41:24.255890 13558 leader_election.cc:302] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e6e59471bcd4c31b213dfe952991ec4, c36cf4f91b244ca2b5b6f8f003a5a033; no voters:
I20260811 18:41:24.256098 14022 raft_consensus.cc:2454] T d503c1292f534b7295ce1c07ea4b5dd0 P 56dd0faaf36046478b6dfddbe800f300 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e6e59471bcd4c31b213dfe952991ec4 in term 1.
I20260811 18:41:24.256155 14448 raft_consensus.cc:2781] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:41:24.256659 14448 raft_consensus.cc:686] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4 [term 1 LEADER]: Becoming Leader. State: Replica: 3e6e59471bcd4c31b213dfe952991ec4, State: Running, Role: LEADER
I20260811 18:41:24.256939 14448 consensus_queue.cc:237] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:24.260092 13381 catalog_manager.cc:5495] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e6e59471bcd4c31b213dfe952991ec4 (127.30.185.129). New cstate: current_term: 1 leader_uuid: "3e6e59471bcd4c31b213dfe952991ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } health_report { overall_health: UNKNOWN } } }
I20260811 18:41:24.306133 14612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:24.296353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54422 (local address 127.30.185.129:37959)
0811 18:41:24.297014 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:24.297021 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:24.297205 (+ 184us) server_negotiation.cc:407] Connection header received
0811 18:41:24.297452 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:24.297458 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:24.297566 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:24.297683 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:41:24.300177 (+ 2494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:24.301111 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:24.302998 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:24.303824 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:24.303977 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:24.304502 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:41:24.304668 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:41:24.304778 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:24.304938 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 18:41:24.305027 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":362,"threads_started":1}
I20260811 18:41:24.309347 14294 raft_consensus.cc:1261] T d503c1292f534b7295ce1c07ea4b5dd0 P c36cf4f91b244ca2b5b6f8f003a5a033 [term 1 FOLLOWER]: Refusing update from remote peer 3e6e59471bcd4c31b213dfe952991ec4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:41:24.309406 14022 raft_consensus.cc:1261] T d503c1292f534b7295ce1c07ea4b5dd0 P 56dd0faaf36046478b6dfddbe800f300 [term 1 FOLLOWER]: Refusing update from remote peer 3e6e59471bcd4c31b213dfe952991ec4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:41:24.310107 14609 consensus_queue.cc:1035] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:41:24.310592 14448 consensus_queue.cc:1035] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:41:24.337046 13381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:41:24.337393 13381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:41:24.344135 13381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877: Sending DeleteTablet for 3 replicas of tablet d503c1292f534b7295ce1c07ea4b5dd0
I20260811 18:41:24.345270 13675 tablet_service.cc:1514] Processing DeleteTablet for tablet d503c1292f534b7295ce1c07ea4b5dd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:41:24 UTC) from {username='oryx', principal='oryx/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:54410
I20260811 18:41:24.345449 14252 tablet_service.cc:1514] Processing DeleteTablet for tablet d503c1292f534b7295ce1c07ea4b5dd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:41:24 UTC) from {username='oryx', principal='oryx/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:52884
I20260811 18:41:24.345754 14563 tablet_replica.cc:330] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4: stopping tablet replica
I20260811 18:41:24.345855 13978 tablet_service.cc:1514] Processing DeleteTablet for tablet d503c1292f534b7295ce1c07ea4b5dd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:41:24 UTC) from {username='oryx', principal='oryx/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:49974
I20260811 18:41:24.345958 14563 raft_consensus.cc:2229] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:41:24.346140 14565 tablet_replica.cc:330] T d503c1292f534b7295ce1c07ea4b5dd0 P 56dd0faaf36046478b6dfddbe800f300: stopping tablet replica
I20260811 18:41:24.346293 14563 raft_consensus.cc:2258] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:41:24.346351 14565 raft_consensus.cc:2229] T d503c1292f534b7295ce1c07ea4b5dd0 P 56dd0faaf36046478b6dfddbe800f300 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:41:24.346526 14568 tablet_replica.cc:330] T d503c1292f534b7295ce1c07ea4b5dd0 P c36cf4f91b244ca2b5b6f8f003a5a033: stopping tablet replica
I20260811 18:41:24.346573 14565 raft_consensus.cc:2258] T d503c1292f534b7295ce1c07ea4b5dd0 P 56dd0faaf36046478b6dfddbe800f300 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:41:24.346743 14568 raft_consensus.cc:2229] T d503c1292f534b7295ce1c07ea4b5dd0 P c36cf4f91b244ca2b5b6f8f003a5a033 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:41:24.346987 14568 raft_consensus.cc:2258] T d503c1292f534b7295ce1c07ea4b5dd0 P c36cf4f91b244ca2b5b6f8f003a5a033 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:41:24.347487 13674 tablet_service.cc:1514] Processing DeleteTablet for tablet d503c1292f534b7295ce1c07ea4b5dd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:41:24 UTC) from {username='oryx', principal='oryx/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:54410
I20260811 18:41:24.348078 14565 ts_tablet_manager.cc:1905] T d503c1292f534b7295ce1c07ea4b5dd0 P 56dd0faaf36046478b6dfddbe800f300: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:41:24.350016 14563 ts_tablet_manager.cc:1905] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:41:24.350701 13373 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:41:24.351388 13373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:41:24.352802 14568 ts_tablet_manager.cc:1905] T d503c1292f534b7295ce1c07ea4b5dd0 P c36cf4f91b244ca2b5b6f8f003a5a033: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:41:24.354848 14565 ts_tablet_manager.cc:1918] T d503c1292f534b7295ce1c07ea4b5dd0 P 56dd0faaf36046478b6dfddbe800f300: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:41:24.354997 14565 log.cc:1196] T d503c1292f534b7295ce1c07ea4b5dd0 P 56dd0faaf36046478b6dfddbe800f300: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/d503c1292f534b7295ce1c07ea4b5dd0
I20260811 18:41:24.355484 14565 ts_tablet_manager.cc:1939] T d503c1292f534b7295ce1c07ea4b5dd0 P 56dd0faaf36046478b6dfddbe800f300: Deleting consensus metadata
I20260811 18:41:24.356611 13353 catalog_manager.cc:4841] TS 56dd0faaf36046478b6dfddbe800f300 (127.30.185.130:40829): tablet d503c1292f534b7295ce1c07ea4b5dd0 (table test-table [id=e743783c988b4550be1ae1f22ee28e87]) successfully deleted
I20260811 18:41:24.359858 13978 tablet_service.cc:1467] Processing CreateTablet for tablet be0365128a63452d9f4d42124eee471f (DEFAULT_TABLE table=test-table [id=122fbeee5b4d471ca4a2ec4dfae65214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:41:24.360008 14252 tablet_service.cc:1467] Processing CreateTablet for tablet be0365128a63452d9f4d42124eee471f (DEFAULT_TABLE table=test-table [id=122fbeee5b4d471ca4a2ec4dfae65214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:41:24.360288 13978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be0365128a63452d9f4d42124eee471f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:24.360466 14252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be0365128a63452d9f4d42124eee471f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:24.360508 14568 ts_tablet_manager.cc:1918] T d503c1292f534b7295ce1c07ea4b5dd0 P c36cf4f91b244ca2b5b6f8f003a5a033: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:41:24.360739 14568 log.cc:1196] T d503c1292f534b7295ce1c07ea4b5dd0 P c36cf4f91b244ca2b5b6f8f003a5a033: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/d503c1292f534b7295ce1c07ea4b5dd0
I20260811 18:41:24.361481 14568 ts_tablet_manager.cc:1939] T d503c1292f534b7295ce1c07ea4b5dd0 P c36cf4f91b244ca2b5b6f8f003a5a033: Deleting consensus metadata
I20260811 18:41:24.361899 13674 tablet_service.cc:1467] Processing CreateTablet for tablet be0365128a63452d9f4d42124eee471f (DEFAULT_TABLE table=test-table [id=122fbeee5b4d471ca4a2ec4dfae65214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:41:24.362350 13674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be0365128a63452d9f4d42124eee471f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:24.366652 14599 tablet_bootstrap.cc:492] T be0365128a63452d9f4d42124eee471f P 56dd0faaf36046478b6dfddbe800f300: Bootstrap starting.
I20260811 18:41:24.368111 13352 catalog_manager.cc:4841] TS c36cf4f91b244ca2b5b6f8f003a5a033 (127.30.185.131:37899): tablet d503c1292f534b7295ce1c07ea4b5dd0 (table test-table [id=e743783c988b4550be1ae1f22ee28e87]) successfully deleted
I20260811 18:41:24.368362 14599 tablet_bootstrap.cc:654] T be0365128a63452d9f4d42124eee471f P 56dd0faaf36046478b6dfddbe800f300: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:24.369867 14598 tablet_bootstrap.cc:492] T be0365128a63452d9f4d42124eee471f P 3e6e59471bcd4c31b213dfe952991ec4: Bootstrap starting.
I20260811 18:41:24.371801 14598 tablet_bootstrap.cc:654] T be0365128a63452d9f4d42124eee471f P 3e6e59471bcd4c31b213dfe952991ec4: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:24.377193 14601 tablet_bootstrap.cc:492] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033: Bootstrap starting.
I20260811 18:41:24.377907 14563 ts_tablet_manager.cc:1918] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:41:24.378158 14563 log.cc:1196] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-0/wal/wals/d503c1292f534b7295ce1c07ea4b5dd0
I20260811 18:41:24.378350 14599 tablet_bootstrap.cc:492] T be0365128a63452d9f4d42124eee471f P 56dd0faaf36046478b6dfddbe800f300: No bootstrap required, opened a new log
I20260811 18:41:24.378542 14599 ts_tablet_manager.cc:1397] T be0365128a63452d9f4d42124eee471f P 56dd0faaf36046478b6dfddbe800f300: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260811 18:41:24.378909 14563 ts_tablet_manager.cc:1939] T d503c1292f534b7295ce1c07ea4b5dd0 P 3e6e59471bcd4c31b213dfe952991ec4: Deleting consensus metadata
I20260811 18:41:24.379266 14601 tablet_bootstrap.cc:654] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:24.379410 14599 raft_consensus.cc:348] T be0365128a63452d9f4d42124eee471f P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:24.379690 14599 raft_consensus.cc:374] T be0365128a63452d9f4d42124eee471f P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:24.379781 14599 raft_consensus.cc:729] T be0365128a63452d9f4d42124eee471f P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56dd0faaf36046478b6dfddbe800f300, State: Initialized, Role: FOLLOWER
I20260811 18:41:24.380066 14599 consensus_queue.cc:260] T be0365128a63452d9f4d42124eee471f P 56dd0faaf36046478b6dfddbe800f300 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:24.380350 13355 catalog_manager.cc:4841] TS 3e6e59471bcd4c31b213dfe952991ec4 (127.30.185.129:37959): tablet d503c1292f534b7295ce1c07ea4b5dd0 (table test-table [id=e743783c988b4550be1ae1f22ee28e87]) successfully deleted
I20260811 18:41:24.380683 14599 ts_tablet_manager.cc:1428] T be0365128a63452d9f4d42124eee471f P 56dd0faaf36046478b6dfddbe800f300: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260811 18:41:24.381367 13355 catalog_manager.cc:4808] TS 3e6e59471bcd4c31b213dfe952991ec4 (127.30.185.129:37959): delete failed for tablet d503c1292f534b7295ce1c07ea4b5dd0 because the tablet was not found. No further retry: Not found: Tablet not found: d503c1292f534b7295ce1c07ea4b5dd0
I20260811 18:41:24.383363 14601 tablet_bootstrap.cc:492] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033: No bootstrap required, opened a new log
I20260811 18:41:24.383579 14601 ts_tablet_manager.cc:1397] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 18:41:24.384657 14601 raft_consensus.cc:348] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:24.385020 14601 raft_consensus.cc:374] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:24.385133 14601 raft_consensus.cc:729] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c36cf4f91b244ca2b5b6f8f003a5a033, State: Initialized, Role: FOLLOWER
I20260811 18:41:24.385493 14601 consensus_queue.cc:260] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:24.386155 14601 ts_tablet_manager.cc:1428] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:41:24.387072 14598 tablet_bootstrap.cc:492] T be0365128a63452d9f4d42124eee471f P 3e6e59471bcd4c31b213dfe952991ec4: No bootstrap required, opened a new log
I20260811 18:41:24.387277 14598 ts_tablet_manager.cc:1397] T be0365128a63452d9f4d42124eee471f P 3e6e59471bcd4c31b213dfe952991ec4: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.009s
I20260811 18:41:24.388120 14598 raft_consensus.cc:348] T be0365128a63452d9f4d42124eee471f P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:24.388414 14598 raft_consensus.cc:374] T be0365128a63452d9f4d42124eee471f P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:24.388509 14598 raft_consensus.cc:729] T be0365128a63452d9f4d42124eee471f P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e6e59471bcd4c31b213dfe952991ec4, State: Initialized, Role: FOLLOWER
I20260811 18:41:24.389415 14598 consensus_queue.cc:260] T be0365128a63452d9f4d42124eee471f P 3e6e59471bcd4c31b213dfe952991ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:24.390103 14598 ts_tablet_manager.cc:1428] T be0365128a63452d9f4d42124eee471f P 3e6e59471bcd4c31b213dfe952991ec4: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260811 18:41:24.556137 14545 raft_consensus.cc:482] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:41:24.556438 14545 raft_consensus.cc:504] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:24.557195 14545 leader_election.cc:288] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e6e59471bcd4c31b213dfe952991ec4 (127.30.185.129:37959), 56dd0faaf36046478b6dfddbe800f300 (127.30.185.130:40829)
I20260811 18:41:24.557822 13716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be0365128a63452d9f4d42124eee471f" candidate_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e6e59471bcd4c31b213dfe952991ec4" is_pre_election: true
I20260811 18:41:24.557905 14022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be0365128a63452d9f4d42124eee471f" candidate_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56dd0faaf36046478b6dfddbe800f300" is_pre_election: true
I20260811 18:41:24.558141 14022 raft_consensus.cc:2454] T be0365128a63452d9f4d42124eee471f P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36cf4f91b244ca2b5b6f8f003a5a033 in term 0.
I20260811 18:41:24.558153 13716 raft_consensus.cc:2454] T be0365128a63452d9f4d42124eee471f P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c36cf4f91b244ca2b5b6f8f003a5a033 in term 0.
I20260811 18:41:24.558609 14145 leader_election.cc:302] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56dd0faaf36046478b6dfddbe800f300, c36cf4f91b244ca2b5b6f8f003a5a033; no voters:
I20260811 18:41:24.558912 14545 raft_consensus.cc:2781] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:41:24.559022 14545 raft_consensus.cc:482] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:41:24.559154 14545 raft_consensus.cc:3037] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:24.561233 14545 raft_consensus.cc:504] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:24.561802 14545 leader_election.cc:288] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033 [CANDIDATE]: Term 1 election: Requested vote from peers 3e6e59471bcd4c31b213dfe952991ec4 (127.30.185.129:37959), 56dd0faaf36046478b6dfddbe800f300 (127.30.185.130:40829)
I20260811 18:41:24.562192 13716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be0365128a63452d9f4d42124eee471f" candidate_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e6e59471bcd4c31b213dfe952991ec4"
I20260811 18:41:24.562278 14022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be0365128a63452d9f4d42124eee471f" candidate_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56dd0faaf36046478b6dfddbe800f300"
I20260811 18:41:24.562417 13716 raft_consensus.cc:3037] T be0365128a63452d9f4d42124eee471f P 3e6e59471bcd4c31b213dfe952991ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:24.562511 14022 raft_consensus.cc:3037] T be0365128a63452d9f4d42124eee471f P 56dd0faaf36046478b6dfddbe800f300 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:24.564489 14022 raft_consensus.cc:2454] T be0365128a63452d9f4d42124eee471f P 56dd0faaf36046478b6dfddbe800f300 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36cf4f91b244ca2b5b6f8f003a5a033 in term 1.
I20260811 18:41:24.564494 13716 raft_consensus.cc:2454] T be0365128a63452d9f4d42124eee471f P 3e6e59471bcd4c31b213dfe952991ec4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c36cf4f91b244ca2b5b6f8f003a5a033 in term 1.
I20260811 18:41:24.564949 14149 leader_election.cc:302] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e6e59471bcd4c31b213dfe952991ec4, c36cf4f91b244ca2b5b6f8f003a5a033; no voters:
I20260811 18:41:24.565356 14545 raft_consensus.cc:2781] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:41:24.565827 14545 raft_consensus.cc:686] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033 [term 1 LEADER]: Becoming Leader. State: Replica: c36cf4f91b244ca2b5b6f8f003a5a033, State: Running, Role: LEADER
I20260811 18:41:24.566092 14545 consensus_queue.cc:237] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } }
I20260811 18:41:24.568909 13373 catalog_manager.cc:5495] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033 reported cstate change: term changed from 0 to 1, leader changed from <none> to c36cf4f91b244ca2b5b6f8f003a5a033 (127.30.185.131). New cstate: current_term: 1 leader_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c36cf4f91b244ca2b5b6f8f003a5a033" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37899 } health_report { overall_health: HEALTHY } } }
I20260811 18:41:24.635116 14639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:24.629770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52912 (local address 127.30.185.131:37899)
0811 18:41:24.630409 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:24.630414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:24.630428 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:41:24.630479 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:24.630483 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:24.630580 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:24.630694 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:41:24.631102 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:24.631866 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:24.633278 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:24.633716 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:24.633859 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:24.634334 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:41:24.634447 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:41:24.634548 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:24.634695 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 18:41:24.634786 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":362,"threads_started":1}
I20260811 18:41:24.644585 14022 raft_consensus.cc:1261] T be0365128a63452d9f4d42124eee471f P 56dd0faaf36046478b6dfddbe800f300 [term 1 FOLLOWER]: Refusing update from remote peer c36cf4f91b244ca2b5b6f8f003a5a033: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:41:24.644695 13716 raft_consensus.cc:1261] T be0365128a63452d9f4d42124eee471f P 3e6e59471bcd4c31b213dfe952991ec4 [term 1 FOLLOWER]: Refusing update from remote peer c36cf4f91b244ca2b5b6f8f003a5a033: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:41:24.645215 14626 consensus_queue.cc:1035] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56dd0faaf36046478b6dfddbe800f300" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:41:24.645605 14545 consensus_queue.cc:1035] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e6e59471bcd4c31b213dfe952991ec4" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:41:24.699564 13380 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:41:24.699838 13380 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:41:24.702843 13380 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 362393199dbd4999947c30aa9a3d6877: Sending DeleteTablet for 3 replicas of tablet be0365128a63452d9f4d42124eee471f
I20260811 18:41:24.703665 13978 tablet_service.cc:1514] Processing DeleteTablet for tablet be0365128a63452d9f4d42124eee471f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:41:24 UTC) from {username='oryx', principal='oryx/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:49974
I20260811 18:41:24.703711 13674 tablet_service.cc:1514] Processing DeleteTablet for tablet be0365128a63452d9f4d42124eee471f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:41:24 UTC) from {username='oryx', principal='oryx/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:54410
I20260811 18:41:24.703980 14565 tablet_replica.cc:330] T be0365128a63452d9f4d42124eee471f P 56dd0faaf36046478b6dfddbe800f300: stopping tablet replica
I20260811 18:41:24.704015 14563 tablet_replica.cc:330] T be0365128a63452d9f4d42124eee471f P 3e6e59471bcd4c31b213dfe952991ec4: stopping tablet replica
I20260811 18:41:24.704183 14563 raft_consensus.cc:2229] T be0365128a63452d9f4d42124eee471f P 3e6e59471bcd4c31b213dfe952991ec4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:41:24.704185 14565 raft_consensus.cc:2229] T be0365128a63452d9f4d42124eee471f P 56dd0faaf36046478b6dfddbe800f300 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:41:24.704406 14563 raft_consensus.cc:2258] T be0365128a63452d9f4d42124eee471f P 3e6e59471bcd4c31b213dfe952991ec4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:41:24.704401 14565 raft_consensus.cc:2258] T be0365128a63452d9f4d42124eee471f P 56dd0faaf36046478b6dfddbe800f300 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:41:24.704692 14252 tablet_service.cc:1514] Processing DeleteTablet for tablet be0365128a63452d9f4d42124eee471f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:41:24 UTC) from {username='oryx', principal='oryx/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:52884
I20260811 18:41:24.704990 14568 tablet_replica.cc:330] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033: stopping tablet replica
I20260811 18:41:24.705150 14568 raft_consensus.cc:2229] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:41:24.705518 14568 raft_consensus.cc:2258] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:41:24.705547 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 13523
I20260811 18:41:24.705804 14565 ts_tablet_manager.cc:1905] T be0365128a63452d9f4d42124eee471f P 56dd0faaf36046478b6dfddbe800f300: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:41:24.706705 14563 ts_tablet_manager.cc:1905] T be0365128a63452d9f4d42124eee471f P 3e6e59471bcd4c31b213dfe952991ec4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:41:24.714457 14568 ts_tablet_manager.cc:1905] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:41:24.714838 14565 ts_tablet_manager.cc:1918] T be0365128a63452d9f4d42124eee471f P 56dd0faaf36046478b6dfddbe800f300: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:41:24.714991 14565 log.cc:1196] T be0365128a63452d9f4d42124eee471f P 56dd0faaf36046478b6dfddbe800f300: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/be0365128a63452d9f4d42124eee471f
I20260811 18:41:24.715469 14565 ts_tablet_manager.cc:1939] T be0365128a63452d9f4d42124eee471f P 56dd0faaf36046478b6dfddbe800f300: Deleting consensus metadata
I20260811 18:41:24.715757 14563 ts_tablet_manager.cc:1918] T be0365128a63452d9f4d42124eee471f P 3e6e59471bcd4c31b213dfe952991ec4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:41:24.715919 14563 log.cc:1196] T be0365128a63452d9f4d42124eee471f P 3e6e59471bcd4c31b213dfe952991ec4: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-0/wal/wals/be0365128a63452d9f4d42124eee471f
I20260811 18:41:24.716418 14563 ts_tablet_manager.cc:1939] T be0365128a63452d9f4d42124eee471f P 3e6e59471bcd4c31b213dfe952991ec4: Deleting consensus metadata
I20260811 18:41:24.716740 13353 catalog_manager.cc:4841] TS 56dd0faaf36046478b6dfddbe800f300 (127.30.185.130:40829): tablet be0365128a63452d9f4d42124eee471f (table test-table [id=122fbeee5b4d471ca4a2ec4dfae65214]) successfully deleted
I20260811 18:41:24.717962 13355 catalog_manager.cc:4841] TS 3e6e59471bcd4c31b213dfe952991ec4 (127.30.185.129:37959): tablet be0365128a63452d9f4d42124eee471f (table test-table [id=122fbeee5b4d471ca4a2ec4dfae65214]) successfully deleted
I20260811 18:41:24.720975 14568 ts_tablet_manager.cc:1918] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:41:24.721144 14568 log.cc:1196] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/be0365128a63452d9f4d42124eee471f
I20260811 18:41:24.721603 14568 ts_tablet_manager.cc:1939] T be0365128a63452d9f4d42124eee471f P c36cf4f91b244ca2b5b6f8f003a5a033: Deleting consensus metadata
I20260811 18:41:24.722652 13352 catalog_manager.cc:4841] TS c36cf4f91b244ca2b5b6f8f003a5a033 (127.30.185.131:37899): tablet be0365128a63452d9f4d42124eee471f (table test-table [id=122fbeee5b4d471ca4a2ec4dfae65214]) successfully deleted
Aug 11 18:41:24 dist-test-slave-z8x9 krb5kdc[13283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.185.129@KRBTEST.COM
I20260811 18:41:24.731479 14612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:24.707071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54430 (local address 127.30.185.129:37959)
0811 18:41:24.707239 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:24.707256 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:24.714470 (+ 7214us) server_negotiation.cc:407] Connection header received
0811 18:41:24.714732 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:24.714738 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:24.714833 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:24.714945 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:24.717582 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:24.718510 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:24.719760 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:24.720169 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:24.720305 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:24.724964 (+ 4659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:24.724983 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:24.725001 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:24.725045 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:24.728650 (+ 3605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:24.729535 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:24.729544 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:24.729546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:24.729727 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:24.730071 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:24.730074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:24.730076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:24.730532 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:24.730824 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:24.731061 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 18:41:24.731235 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:41:24.732160 13808 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:24.862111 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 13523
I20260811 18:41:24.896548 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 13826
Aug 11 18:41:24 dist-test-slave-z8x9 krb5kdc[13283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.185.130@KRBTEST.COM
I20260811 18:41:24.913378 14657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:24.897472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49994 (local address 127.30.185.130:40829)
0811 18:41:24.898017 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:24.898022 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:24.898038 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:41:24.898090 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:24.898094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:24.898151 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:24.898338 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:24.899630 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:24.900375 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:24.902130 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:24.902578 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:24.902724 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:24.907131 (+ 4407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:24.907147 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:24.907163 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:24.907204 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:24.910454 (+ 3250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:24.911195 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:24.911203 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:24.911205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:24.911350 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:24.911740 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:24.911746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:24.911749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:24.912096 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:24.912425 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:24.912713 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 18:41:24.912952 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":304,"threads_started":1}
I20260811 18:41:24.914103 14096 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:25.043206 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 13826
I20260811 18:41:25.071751 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 14114
Aug 11 18:41:25 dist-test-slave-z8x9 krb5kdc[13283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.185.131@KRBTEST.COM
I20260811 18:41:25.087733 14639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:25.072508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52926 (local address 127.30.185.131:37899)
0811 18:41:25.072680 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:25.072684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:25.072702 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:41:25.072882 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:25.072887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:25.072951 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:25.073050 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:25.074420 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:25.075154 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:25.076549 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:25.077036 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:25.077157 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:25.081803 (+ 4646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:25.081817 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:25.081835 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:25.081869 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:25.084967 (+ 3098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:25.085847 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:25.085855 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:25.085857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:25.085987 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:25.086410 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:25.086413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:25.086415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:25.086787 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:25.087037 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:25.087299 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 18:41:25.087487 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 18:41:25.088471 14372 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:25.217864 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 14114
I20260811 18:41:25.252398 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 13310
I20260811 18:41:25.253722 13463 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:25.375928 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 13310
2026-08-11T18:41:25Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6563 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:41:25 dist-test-slave-z8x9 krb5kdc[14665](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:41:25 dist-test-slave-z8x9 krb5kdc[14665](info): set up 2 sockets
Aug 11 18:41:25 dist-test-slave-z8x9 krb5kdc[14665](info): commencing operation
krb5kdc: starting...
W20260811 18:41:26.791906 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.355s user 0.005s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:41:26 dist-test-slave-z8x9 krb5kdc[14665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:41:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:41:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:41:27.108661 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:44047
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:45179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:44047
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:41:27.455482 14690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:27.455788 14690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:27.455951 14690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:27.463267 14690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:41:27.463384 14690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:27.463485 14690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:27.463588 14690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:41:27.463677 14690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:41:27.473968 14690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:45179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:44047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:44047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:27.475346 14690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:27.476665 14690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:27.488860 14702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:27.488766 14705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:27.488822 14713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:27.490553 14690 server_base.cc:1034] running on GCE node
I20260811 18:41:27.491393 14690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:27.492843 14690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:27.494076 14690 hybrid_clock.cc:648] HybridClock initialized: now 1786473687494036 us; error 52 us; skew 500 ppm
I20260811 18:41:27.494454 14690 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T18:41:27Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2184 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:41:27 dist-test-slave-z8x9 krb5kdc[14722](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:41:27 dist-test-slave-z8x9 krb5kdc[14722](info): set up 2 sockets
Aug 11 18:41:27 dist-test-slave-z8x9 krb5kdc[14722](info): commencing operation
krb5kdc: starting...
W20260811 18:41:28.989156 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.368s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:41:29 dist-test-slave-z8x9 krb5kdc[14722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:41:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:41:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:41:29.294111 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:41149
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:33105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:41149
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:41:29.637964 14747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:29.638252 14747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:29.638439 14747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:29.645812 14747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:41:29.645920 14747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:29.646054 14747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:29.646152 14747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:41:29.646237 14747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:41:29.656144 14747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:33105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:41149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:41149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:29.657509 14747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:29.658835 14747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:29.670849 14755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:29.670964 14758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:29.670955 14754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:29.671844 14747 server_base.cc:1034] running on GCE node
I20260811 18:41:29.672679 14747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:29.674173 14747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:29.675422 14747 hybrid_clock.cc:648] HybridClock initialized: now 1786473689675384 us; error 48 us; skew 500 ppm
I20260811 18:41:29.675781 14747 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:29 dist-test-slave-z8x9 krb5kdc[14722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473689, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:29.679085 14747 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:41:29.680576 14747 webserver.cc:460] Webserver started at http://127.30.185.190:42791/ using document root <none> and password file <none>
I20260811 18:41:29.681339 14747 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:29.681502 14747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:29.681824 14747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:29.684775 14747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "1304b7f850664354b7b70496762d69d8"
format_stamp: "Formatted at 2026-08-11 18:41:29 on dist-test-slave-z8x9"
server_key: "e089a16a82c68e1a3114cf99ce21790c"
server_key_iv: "813364ad7ba6afc48bfa24bfa19e8ba5"
server_key_version: "encryptionkey@0"
I20260811 18:41:29.685595 14747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "1304b7f850664354b7b70496762d69d8"
format_stamp: "Formatted at 2026-08-11 18:41:29 on dist-test-slave-z8x9"
server_key: "e089a16a82c68e1a3114cf99ce21790c"
server_key_iv: "813364ad7ba6afc48bfa24bfa19e8ba5"
server_key_version: "encryptionkey@0"
I20260811 18:41:29.690790 14747 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 18:41:29.694237 14775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:29.695537 14747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:41:29.695748 14747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "1304b7f850664354b7b70496762d69d8"
format_stamp: "Formatted at 2026-08-11 18:41:29 on dist-test-slave-z8x9"
server_key: "e089a16a82c68e1a3114cf99ce21790c"
server_key_iv: "813364ad7ba6afc48bfa24bfa19e8ba5"
server_key_version: "encryptionkey@0"
I20260811 18:41:29.695956 14747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:29.716964 14747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:29.721920 14747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:29.722231 14747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:29.741314 14747 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:41149
I20260811 18:41:29.741335 14905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:41149 every 8 connection(s)
I20260811 18:41:29.742787 14747 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:41:29.744446 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 14747
I20260811 18:41:29.744694 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:41:29.745621 31462 external_mini_cluster.cc:1428] Setting key caa38b40a8eca4301b3ee5b3e40b5326
I20260811 18:41:29.747510 14906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:29.758970 14906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8: Bootstrap starting.
Aug 11 18:41:29 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:41:29.762305 14906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:29.763382 14906 log.cc:824] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8: Log is configured to *not* fsync() on all Append() calls
I20260811 18:41:29.766310 14906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8: No bootstrap required, opened a new log
I20260811 18:41:29.768581 14912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:29.747903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60070 (local address 127.30.185.190:41149)
0811 18:41:29.748679 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:29.748691 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:29.748732 (+ 41us) server_negotiation.cc:407] Connection header received
0811 18:41:29.750463 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:29.750485 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:29.750889 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:29.751302 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:29.752898 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:29.754264 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:29.755551 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:29.756260 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:29.756488 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:29.760689 (+ 4201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:29.760698 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:29.760712 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:29.760750 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:29.764001 (+ 3251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:29.764742 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:29.764753 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:29.764764 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:29.764951 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:29.765424 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:29.765430 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:29.765433 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:29.765975 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:29.766306 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:29.766809 (+ 503us) server_negotiation.cc:299] Negotiation successful
0811 18:41:29.767278 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":370,"threads_started":1}
I20260811 18:41:29.775015 14906 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1304b7f850664354b7b70496762d69d8" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41149 } }
I20260811 18:41:29.775369 14906 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:29.775447 14906 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1304b7f850664354b7b70496762d69d8, State: Initialized, Role: FOLLOWER
I20260811 18:41:29.775923 14906 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1304b7f850664354b7b70496762d69d8" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41149 } }
I20260811 18:41:29.776149 14906 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:41:29.776348 14906 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:41:29.776587 14906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:29.779017 14906 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1304b7f850664354b7b70496762d69d8" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41149 } }
I20260811 18:41:29.779603 14906 leader_election.cc:302] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1304b7f850664354b7b70496762d69d8; no voters:
I20260811 18:41:29.780198 14906 leader_election.cc:288] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:41:29.780351 14916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:41:29.781050 14916 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [term 1 LEADER]: Becoming Leader. State: Replica: 1304b7f850664354b7b70496762d69d8, State: Running, Role: LEADER
I20260811 18:41:29.781643 14916 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1304b7f850664354b7b70496762d69d8" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41149 } }
I20260811 18:41:29.782339 14906 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:41:29.784358 14920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1304b7f850664354b7b70496762d69d8. Latest consensus state: current_term: 1 leader_uuid: "1304b7f850664354b7b70496762d69d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1304b7f850664354b7b70496762d69d8" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41149 } } }
I20260811 18:41:29.784770 14920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:29.784770 14918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1304b7f850664354b7b70496762d69d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1304b7f850664354b7b70496762d69d8" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41149 } } }
I20260811 18:41:29.785418 14918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:29.787567 14923 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:41:29.792912 14923 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:41:29.802590 14923 catalog_manager.cc:1347] Generated new cluster ID: 1a37ed272ccd49348e1eab4fd01d7d0b
I20260811 18:41:29.802719 14923 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:41:29.832283 14923 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:41:29.833828 14923 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:41:29.842336 14923 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8: Generated new TSK 0
I20260811 18:41:29.843428 14923 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.185.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
I20260811 18:41:29.925515 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41149
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:33105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:41:30.282989 14955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:30.283286 14955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:30.283447 14955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:30.291213 14955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:30.291333 14955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:30.291530 14955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:41:30.301971 14955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:33105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:30.303412 14955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:30.304859 14955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:30.318585 14965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:30.319388 14966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:30.322714 14975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:30.326820 14955 server_base.cc:1034] running on GCE node
I20260811 18:41:30.327644 14955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:30.328754 14955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:30.329988 14955 hybrid_clock.cc:648] HybridClock initialized: now 1786473690329927 us; error 65 us; skew 500 ppm
I20260811 18:41:30.330327 14955 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:30 dist-test-slave-z8x9 krb5kdc[14722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473690, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:30.333844 14955 init.cc:377] Logged in from keytab as kudu/127.30.185.129@KRBTEST.COM (short username kudu)
I20260811 18:41:30.335542 14955 webserver.cc:460] Webserver started at http://127.30.185.129:43395/ using document root <none> and password file <none>
I20260811 18:41:30.336308 14955 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:30.336463 14955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:30.336796 14955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:30.339865 14955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "8387bbf8f2ac4862a601a861588b9cc9"
format_stamp: "Formatted at 2026-08-11 18:41:30 on dist-test-slave-z8x9"
server_key: "152b523d40c89027a6f846b2ece3b511"
server_key_iv: "db6ad8c3289d774f80f111810e935571"
server_key_version: "encryptionkey@0"
I20260811 18:41:30.340677 14955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "8387bbf8f2ac4862a601a861588b9cc9"
format_stamp: "Formatted at 2026-08-11 18:41:30 on dist-test-slave-z8x9"
server_key: "152b523d40c89027a6f846b2ece3b511"
server_key_iv: "db6ad8c3289d774f80f111810e935571"
server_key_version: "encryptionkey@0"
I20260811 18:41:30.346455 14955 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 18:41:30.350072 14990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:30.351405 14955 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:41:30.351608 14955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "8387bbf8f2ac4862a601a861588b9cc9"
format_stamp: "Formatted at 2026-08-11 18:41:30 on dist-test-slave-z8x9"
server_key: "152b523d40c89027a6f846b2ece3b511"
server_key_iv: "db6ad8c3289d774f80f111810e935571"
server_key_version: "encryptionkey@0"
I20260811 18:41:30.351825 14955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:30.401005 14955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:30.406011 14955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:30.406352 14955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:30.407405 14955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:30.408934 14955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:30.409045 14955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:30.409230 14955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:30.409324 14955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 18:41:30.444412 14955 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:43575
I20260811 18:41:30.444460 15206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:43575 every 8 connection(s)
I20260811 18:41:30.445835 14955 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:41:30.447690 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 14955
I20260811 18:41:30.447947 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:41:30.448644 31462 external_mini_cluster.cc:1428] Setting key 3f0178176ae2ba0d8cd26c98c6c99f3b
Aug 11 18:41:30 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473690, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:41:30.470101 15215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:30.448817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:48441 (local address 127.30.185.190:41149)
0811 18:41:30.449364 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:30.449370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:30.450406 (+ 1036us) server_negotiation.cc:407] Connection header received
0811 18:41:30.453279 (+ 2873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:30.453284 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:30.453351 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:30.453452 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:30.455654 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:30.456627 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:30.459190 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:30.459582 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:30.459685 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:30.464084 (+ 4399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:30.464090 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:30.464094 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:30.464127 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:30.466872 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:30.467637 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:30.467650 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:30.467652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:30.467786 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:30.468248 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:30.468254 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:30.468256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:30.468523 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:30.468685 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:30.469484 (+ 799us) server_negotiation.cc:299] Negotiation successful
0811 18:41:30.469726 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":253,"threads_started":1}
I20260811 18:41:30.471096 15217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:30.449486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.129:48441)
0811 18:41:30.450256 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:30.450296 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:30.452911 (+ 2615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:30.453758 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:30.453773 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:30.454592 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:30.455448 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:30.455476 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:30.456806 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:30.456814 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:30.459038 (+ 2224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:30.459058 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:30.459822 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:30.459832 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:30.460185 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:30.460958 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:30.461000 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:30.463833 (+ 2833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:30.467023 (+ 3190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:30.467033 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:30.467050 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:30.467500 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:30.467903 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:30.467908 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:30.467912 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:30.468117 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:30.468660 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:30.468669 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:30.469027 (+ 358us) client_negotiation.cc:769] Sending connection context
0811 18:41:30.469357 (+ 330us) client_negotiation.cc:240] Negotiation successful
0811 18:41:30.469918 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":352,"threads_started":1}
I20260811 18:41:30.472890 15208 heartbeater.cc:347] Connected to a master server at 127.30.185.190:41149
I20260811 18:41:30.473282 15208 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:30.474004 15208 heartbeater.cc:511] Master 127.30.185.190:41149 requested a full tablet report, sending...
I20260811 18:41:30.476281 14815 ts_manager.cc:194] Registered new tserver with Master: 8387bbf8f2ac4862a601a861588b9cc9 (127.30.185.129:43575)
I20260811 18:41:30.477854 14815 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:48441
WARNING: no policy specified for kudu/127.30.185.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
I20260811 18:41:30.514941 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41149
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:33105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:41:30.864763 15220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:30.865132 15220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:30.865329 15220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:30.873143 15220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:30.873368 15220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:30.873546 15220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:41:30.883872 15220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:33105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:30.885344 15220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:30.886730 15220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:30.901386 15243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:30.901751 15253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:30.901316 15242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:30.902503 15220 server_base.cc:1034] running on GCE node
I20260811 18:41:30.903239 15220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:30.904245 15220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:30.905524 15220 hybrid_clock.cc:648] HybridClock initialized: now 1786473690905449 us; error 64 us; skew 500 ppm
I20260811 18:41:30.905915 15220 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:30 dist-test-slave-z8x9 krb5kdc[14722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473690, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:30.909257 15220 init.cc:377] Logged in from keytab as kudu/127.30.185.130@KRBTEST.COM (short username kudu)
I20260811 18:41:30.910849 15220 webserver.cc:460] Webserver started at http://127.30.185.130:43751/ using document root <none> and password file <none>
I20260811 18:41:30.911643 15220 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:30.911803 15220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:30.912132 15220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:30.915277 15220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "58f60e97098943abbdde5a8e8f3fc3a0"
format_stamp: "Formatted at 2026-08-11 18:41:30 on dist-test-slave-z8x9"
server_key: "59afaad63820d88237d3caf2d36a9b98"
server_key_iv: "c7c67ac38cd88fdb817d71ba3d776549"
server_key_version: "encryptionkey@0"
I20260811 18:41:30.916129 15220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "58f60e97098943abbdde5a8e8f3fc3a0"
format_stamp: "Formatted at 2026-08-11 18:41:30 on dist-test-slave-z8x9"
server_key: "59afaad63820d88237d3caf2d36a9b98"
server_key_iv: "c7c67ac38cd88fdb817d71ba3d776549"
server_key_version: "encryptionkey@0"
I20260811 18:41:30.922273 15220 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 18:41:30.926174 15271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:30.927572 15220 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:41:30.927765 15220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "58f60e97098943abbdde5a8e8f3fc3a0"
format_stamp: "Formatted at 2026-08-11 18:41:30 on dist-test-slave-z8x9"
server_key: "59afaad63820d88237d3caf2d36a9b98"
server_key_iv: "c7c67ac38cd88fdb817d71ba3d776549"
server_key_version: "encryptionkey@0"
I20260811 18:41:30.927994 15220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:30.949069 15220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:30.953472 15220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:30.953792 15220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:30.954854 15220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:30.956427 15220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:30.956509 15220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:30.956600 15220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:30.956645 15220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:30.991384 15220 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:37627
I20260811 18:41:30.991473 15474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:37627 every 8 connection(s)
I20260811 18:41:30.992759 15220 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:41:30.995034 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 15220
I20260811 18:41:30.995265 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:41:30.995817 31462 external_mini_cluster.cc:1428] Setting key 738580fc120af2a81df9e0d8f940b1b2
Aug 11 18:41:31 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473690, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:41:31.018003 15481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:30.995524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:55137 (local address 127.30.185.190:41149)
0811 18:41:30.996070 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:30.996078 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:30.997296 (+ 1218us) server_negotiation.cc:407] Connection header received
0811 18:41:31.000054 (+ 2758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:31.000065 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:31.000137 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:31.000227 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:31.002544 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:31.003288 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:31.005989 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:31.006466 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:31.006579 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:31.011470 (+ 4891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:31.011475 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:31.011479 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:31.011518 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:31.014486 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:31.015377 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:31.015382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:31.015384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:31.015526 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:31.016023 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:31.016027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:31.016029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:31.016324 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:31.016501 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:31.017337 (+ 836us) server_negotiation.cc:299] Negotiation successful
0811 18:41:31.017599 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":262,"threads_started":1}
I20260811 18:41:31.019131 15482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:30.996166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.130:55137)
0811 18:41:30.997076 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:30.997116 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:30.999652 (+ 2536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:31.000545 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:31.000559 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:31.001457 (+ 898us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:31.002347 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:31.002378 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:31.003467 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:31.003475 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:31.005792 (+ 2317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:31.005815 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:31.006757 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:31.006769 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:31.007169 (+ 400us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:31.008057 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:31.008101 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:31.011143 (+ 3042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:31.014669 (+ 3526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:31.014680 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:31.014700 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:31.015210 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:31.015653 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:31.015659 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:31.015662 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:31.015883 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:31.016475 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:31.016484 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:31.016886 (+ 402us) client_negotiation.cc:769] Sending connection context
0811 18:41:31.017234 (+ 348us) client_negotiation.cc:240] Negotiation successful
0811 18:41:31.017851 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":334,"threads_started":1}
I20260811 18:41:31.020989 15476 heartbeater.cc:347] Connected to a master server at 127.30.185.190:41149
I20260811 18:41:31.021355 15476 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:31.022145 15476 heartbeater.cc:511] Master 127.30.185.190:41149 requested a full tablet report, sending...
I20260811 18:41:31.023856 14815 ts_manager.cc:194] Registered new tserver with Master: 58f60e97098943abbdde5a8e8f3fc3a0 (127.30.185.130:37627)
I20260811 18:41:31.025346 14815 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:55137
WARNING: no policy specified for kudu/127.30.185.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
I20260811 18:41:31.065059 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41149
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:33105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:41:31.418448 15487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:31.418784 15487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:31.418975 15487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:31.426534 15487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:31.426664 15487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:31.426913 15487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:41:31.437235 15487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:33105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:31.438598 15487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:31.439953 15487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:31.454458 15499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:31.454422 15498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:31.454555 15502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:31.455490 15487 server_base.cc:1034] running on GCE node
I20260811 18:41:31.456266 15487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:31.457376 15487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:31.458614 15487 hybrid_clock.cc:648] HybridClock initialized: now 1786473691458569 us; error 57 us; skew 500 ppm
I20260811 18:41:31.458961 15487 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:31 dist-test-slave-z8x9 krb5kdc[14722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473691, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:31.462483 15487 init.cc:377] Logged in from keytab as kudu/127.30.185.131@KRBTEST.COM (short username kudu)
I20260811 18:41:31.463940 15487 webserver.cc:460] Webserver started at http://127.30.185.131:44121/ using document root <none> and password file <none>
I20260811 18:41:31.464612 15487 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:31.464715 15487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:31.464994 15487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:31.468165 15487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "d3a3afca517c4cb784cea6a8aa5275e8"
format_stamp: "Formatted at 2026-08-11 18:41:31 on dist-test-slave-z8x9"
server_key: "37f4ccf81ed39e27cbac2f198865ac6a"
server_key_iv: "7cda6a920063a5e5c5b11ae934146c03"
server_key_version: "encryptionkey@0"
I20260811 18:41:31.469070 15487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "d3a3afca517c4cb784cea6a8aa5275e8"
format_stamp: "Formatted at 2026-08-11 18:41:31 on dist-test-slave-z8x9"
server_key: "37f4ccf81ed39e27cbac2f198865ac6a"
server_key_iv: "7cda6a920063a5e5c5b11ae934146c03"
server_key_version: "encryptionkey@0"
I20260811 18:41:31.475361 15487 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260811 18:41:31.479104 15528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:31.480505 15487 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:41:31.480690 15487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "d3a3afca517c4cb784cea6a8aa5275e8"
format_stamp: "Formatted at 2026-08-11 18:41:31 on dist-test-slave-z8x9"
server_key: "37f4ccf81ed39e27cbac2f198865ac6a"
server_key_iv: "7cda6a920063a5e5c5b11ae934146c03"
server_key_version: "encryptionkey@0"
I20260811 18:41:31.480911 15487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:31.481117 15208 heartbeater.cc:503] Master 127.30.185.190:41149 was elected leader, sending a full tablet report...
I20260811 18:41:31.495837 15487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:31.500946 15487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:31.501314 15487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:31.502370 15487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:31.503799 15487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:31.503886 15487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:31.504063 15487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:31.504122 15487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:31.539656 15487 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:33601
I20260811 18:41:31.539706 15719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:33601 every 8 connection(s)
I20260811 18:41:31.541025 15487 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:41:31.547478 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 15487
I20260811 18:41:31.547706 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:41:31.548170 31462 external_mini_cluster.cc:1428] Setting key 1ddee6d234f9b40de1860533a24f8640
Aug 11 18:41:31 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473691, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:41:31.562512 15724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:31.543803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:50397 (local address 127.30.185.190:41149)
0811 18:41:31.544742 (+ 939us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:31.544747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:31.545512 (+ 765us) server_negotiation.cc:407] Connection header received
0811 18:41:31.547381 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:31.547387 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:31.547482 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:31.547608 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:31.549401 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:31.550473 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:31.552188 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:31.552606 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:31.552748 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:31.556387 (+ 3639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:31.556393 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:31.556396 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:31.556430 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:31.559221 (+ 2791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:31.560009 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:31.560014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:31.560016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:31.560146 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:31.560613 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:31.560623 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:31.560625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:31.560898 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:31.561069 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:31.561885 (+ 816us) server_negotiation.cc:299] Negotiation successful
0811 18:41:31.562100 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":820,"thread_start_us":699,"threads_started":1}
I20260811 18:41:31.563875 15727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:31.544495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.131:50397)
0811 18:41:31.545348 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:31.545398 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:31.547096 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:31.547894 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:31.547902 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:31.548472 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:31.549208 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:31.549242 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:31.550645 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:31.550650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:31.552042 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:31.552055 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:31.552841 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:31.552848 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:31.553086 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:31.553879 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:31.553902 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:31.556149 (+ 2247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:31.559384 (+ 3235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:31.559393 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:31.559412 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:31.559850 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:31.560263 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:31.560269 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:31.560272 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:31.560485 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:31.561033 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:31.561041 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:31.561481 (+ 440us) client_negotiation.cc:769] Sending connection context
0811 18:41:31.561772 (+ 291us) client_negotiation.cc:240] Negotiation successful
0811 18:41:31.562334 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":292,"threads_started":1}
I20260811 18:41:31.565395 15720 heartbeater.cc:347] Connected to a master server at 127.30.185.190:41149
I20260811 18:41:31.565735 15720 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:31.566497 15720 heartbeater.cc:511] Master 127.30.185.190:41149 requested a full tablet report, sending...
I20260811 18:41:31.567938 14815 ts_manager.cc:194] Registered new tserver with Master: d3a3afca517c4cb784cea6a8aa5275e8 (127.30.185.131:33601)
I20260811 18:41:31.568776 14815 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:50397
I20260811 18:41:31.576522 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.185.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.185.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.185.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.129.keytab.
Entry for principal oryx/127.30.185.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.129.keytab.
WARNING: no policy specified for oryx/127.30.185.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.185.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.185.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.130.keytab.
Entry for principal oryx/127.30.185.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.130.keytab.
WARNING: no policy specified for oryx/127.30.185.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.185.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.185.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.131.keytab.
Entry for principal oryx/127.30.185.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.131.keytab.
I20260811 18:41:31.770093 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 14955
Aug 11 18:41:31 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.129@KRBTEST.COM
I20260811 18:41:31.787324 15743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:31.771252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50012 (local address 127.30.185.129:43575)
0811 18:41:31.771924 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:31.771932 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:31.771963 (+ 31us) server_negotiation.cc:407] Connection header received
0811 18:41:31.772123 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:31.772130 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:31.772353 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:31.772508 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:31.774049 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:31.775114 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:31.776488 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:31.777059 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:31.777248 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:31.781335 (+ 4087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:31.781347 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:31.781368 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:31.781407 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:31.784514 (+ 3107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:31.785348 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:31.785356 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:31.785358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:31.785500 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:31.785959 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:31.785963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:31.785965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:31.786314 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:31.786575 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:31.786826 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 18:41:31.787038 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":331,"threads_started":1}
I20260811 18:41:31.788707 15195 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:31.873749 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 14955
I20260811 18:41:31.901463 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 15220
Aug 11 18:41:31 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.130@KRBTEST.COM
I20260811 18:41:31.918087 15748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:31.902387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48184 (local address 127.30.185.130:37627)
0811 18:41:31.902965 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:31.902972 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:31.903000 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:41:31.903064 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:31.903071 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:31.903290 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:31.903438 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:31.904671 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:31.905768 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:31.907024 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:31.907587 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:31.907687 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:31.912155 (+ 4468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:31.912169 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:31.912185 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:31.912224 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:31.915250 (+ 3026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:31.915979 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:31.915987 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:31.915989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:31.916124 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:31.916465 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:31.916468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:31.916470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:31.916833 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:31.917084 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:31.917544 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 18:41:31.917770 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":303,"threads_started":1}
I20260811 18:41:31.919468 15466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:32.009379 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 15220
I20260811 18:41:32.036669 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 15487
Aug 11 18:41:32 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.131@KRBTEST.COM
I20260811 18:41:32.054344 15752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:32.037901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48808 (local address 127.30.185.131:33601)
0811 18:41:32.038406 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:32.038413 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:32.038442 (+ 29us) server_negotiation.cc:407] Connection header received
0811 18:41:32.038501 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:32.038508 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:32.038719 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:32.038902 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:32.040319 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:32.041444 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:32.042787 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:32.043358 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:32.043511 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:32.048199 (+ 4688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:32.048211 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:32.048226 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:32.048296 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:32.051499 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:32.052194 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:32.052201 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:32.052203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:32.052333 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:32.052697 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:32.052700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:32.052703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:32.053057 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:32.053428 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:32.053686 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 18:41:32.053883 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":237,"threads_started":1}
I20260811 18:41:32.055729 15712 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:32.142445 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 15487
I20260811 18:41:32.172822 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 14747
I20260811 18:41:32.174253 14898 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:32.281111 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 14747
I20260811 18:41:32.317785 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:41149
--webserver_interface=127.30.185.190
--webserver_port=42791
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:33105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:41149
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:41:32.667488 15756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:32.667907 15756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:32.668104 15756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:32.676494 15756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:41:32.676611 15756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:32.676712 15756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:32.676819 15756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:41:32.676888 15756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:41:32.687053 15756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:33105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:41149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:41149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=42791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:32.688416 15756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:32.689857 15756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:32.702433 15765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:32.702672 15769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:32.702698 15767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:32.703336 15756 server_base.cc:1034] running on GCE node
I20260811 18:41:32.704147 15756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:32.705641 15756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:32.706845 15756 hybrid_clock.cc:648] HybridClock initialized: now 1786473692706808 us; error 30 us; skew 500 ppm
I20260811 18:41:32.707211 15756 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:32 dist-test-slave-z8x9 krb5kdc[14722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473692, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:32.713686 15756 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:41:32.715359 15756 webserver.cc:460] Webserver started at http://127.30.185.190:42791/ using document root <none> and password file <none>
I20260811 18:41:32.716061 15756 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:32.716187 15756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:32.722103 15756 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:41:32.725042 15782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:32.726485 15756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:41:32.726689 15756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "1304b7f850664354b7b70496762d69d8"
format_stamp: "Formatted at 2026-08-11 18:41:29 on dist-test-slave-z8x9"
server_key: "e089a16a82c68e1a3114cf99ce21790c"
server_key_iv: "813364ad7ba6afc48bfa24bfa19e8ba5"
server_key_version: "encryptionkey@0"
I20260811 18:41:32.727758 15756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:32.750180 15756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:32.755008 15756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:32.755326 15756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:32.775058 15756 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:41149
I20260811 18:41:32.775076 15879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:41149 every 8 connection(s)
I20260811 18:41:32.776722 15756 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:41:32.779117 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 15756
I20260811 18:41:32.779937 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:43575
--local_ip_for_outbound_sockets=127.30.185.129
--tserver_master_addrs=127.30.185.190:41149
--webserver_port=43395
--webserver_interface=127.30.185.129
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:33105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.185.129
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.129.keytab with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:41:32.783864 15882 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:41:32.787158 15882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8: Bootstrap starting.
I20260811 18:41:32.806697 15882 log.cc:824] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8: Log is configured to *not* fsync() on all Append() calls
I20260811 18:41:32.816005 15882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:41:32.816587 15882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8: Bootstrap complete.
I20260811 18:41:32.826413 15882 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1304b7f850664354b7b70496762d69d8" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41149 } }
I20260811 18:41:32.827415 15882 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1304b7f850664354b7b70496762d69d8, State: Initialized, Role: FOLLOWER
I20260811 18:41:32.828087 15882 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1304b7f850664354b7b70496762d69d8" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41149 } }
I20260811 18:41:32.828366 15882 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:41:32.828513 15882 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:41:32.828715 15882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:41:32.833141 15882 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1304b7f850664354b7b70496762d69d8" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41149 } }
I20260811 18:41:32.833725 15882 leader_election.cc:302] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1304b7f850664354b7b70496762d69d8; no voters:
I20260811 18:41:32.834403 15882 leader_election.cc:288] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 18:41:32.834549 15889 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:41:32.836905 15882 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:41:32.841777 15889 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [term 2 LEADER]: Becoming Leader. State: Replica: 1304b7f850664354b7b70496762d69d8, State: Running, Role: LEADER
I20260811 18:41:32.842511 15889 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1304b7f850664354b7b70496762d69d8" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41149 } }
I20260811 18:41:32.845395 15890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1304b7f850664354b7b70496762d69d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1304b7f850664354b7b70496762d69d8" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41149 } } }
I20260811 18:41:32.845881 15890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:32.846518 15892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1304b7f850664354b7b70496762d69d8. Latest consensus state: current_term: 2 leader_uuid: "1304b7f850664354b7b70496762d69d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1304b7f850664354b7b70496762d69d8" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41149 } } }
I20260811 18:41:32.846875 15892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:32.850361 15908 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:41:32.856971 15908 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:41:32.859141 15908 catalog_manager.cc:1259] Loaded cluster ID: 1a37ed272ccd49348e1eab4fd01d7d0b
I20260811 18:41:32.859253 15908 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:41:32.863494 15908 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:41:32.864959 15908 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1304b7f850664354b7b70496762d69d8: Loaded TSK: 0
I20260811 18:41:32.866647 15908 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 18:41:33.139354 15886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:33.139642 15886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:33.139784 15886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:33.147641 15886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:33.147840 15886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:33.148056 15886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:41:33.159158 15886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:33105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.129.keytab
--principal=oryx/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:43575
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=43395
--tserver_master_addrs=127.30.185.190:41149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:33.160727 15886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:33.162233 15886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:33.177628 15936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:33.177963 15932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:33.177868 15938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:33.179435 15886 server_base.cc:1034] running on GCE node
I20260811 18:41:33.180272 15886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:33.181483 15886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:33.182731 15886 hybrid_clock.cc:648] HybridClock initialized: now 1786473693182681 us; error 52 us; skew 500 ppm
I20260811 18:41:33.183089 15886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:33 dist-test-slave-z8x9 krb5kdc[14722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473693, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:33.186904 15886 init.cc:377] Logged in from keytab as oryx/127.30.185.129@KRBTEST.COM (short username oryx)
I20260811 18:41:33.188966 15886 webserver.cc:460] Webserver started at http://127.30.185.129:43395/ using document root <none> and password file <none>
I20260811 18:41:33.189872 15886 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:33.190065 15886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:33.196082 15886 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 18:41:33.199353 15955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:33.200989 15886 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:41:33.201239 15886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "8387bbf8f2ac4862a601a861588b9cc9"
format_stamp: "Formatted at 2026-08-11 18:41:30 on dist-test-slave-z8x9"
server_key: "152b523d40c89027a6f846b2ece3b511"
server_key_iv: "db6ad8c3289d774f80f111810e935571"
server_key_version: "encryptionkey@0"
I20260811 18:41:33.202323 15886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:33.214717 15886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:33.220055 15886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:33.220443 15886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:33.221571 15886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:33.222987 15886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:33.223079 15886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:33.223153 15886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:33.223213 15886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:33.260565 15886 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:43575
I20260811 18:41:33.260713 16177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:43575 every 8 connection(s)
I20260811 18:41:33.262192 15886 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:41:33.270880 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 15886
I20260811 18:41:33.271535 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:37627
--local_ip_for_outbound_sockets=127.30.185.130
--tserver_master_addrs=127.30.185.190:41149
--webserver_port=43751
--webserver_interface=127.30.185.130
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:33105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.185.130
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.130.keytab with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:41:33 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:33 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:33.281457 16181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:33.265530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.129:41123)
0811 18:41:33.266394 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:33.266449 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:33.268397 (+ 1948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:33.269894 (+ 1497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:33.269904 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:33.270522 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:33.271105 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:33.271125 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.272810 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.272815 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:33.274074 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:33.274086 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.275126 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.275134 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:33.275404 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:33.276147 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:33.276169 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:33.280117 (+ 3948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":372,"threads_started":1}
W20260811 18:41:33.281535 16183 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:33.265562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:41123 (local address 127.30.185.190:41149)
0811 18:41:33.267308 (+ 1746us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:33.267324 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:33.267383 (+ 59us) server_negotiation.cc:407] Connection header received
0811 18:41:33.268953 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:33.268966 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:33.269378 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:33.269728 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:33.271306 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.272636 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.274238 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.274879 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.275078 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:33.280477 (+ 5399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.129:41123: BlockingRecv error: recv got EOF from 127.30.185.129:41123 (error 108)
Metrics: {"server-negotiator.queue_time_us":1488,"thread_start_us":1291,"threads_started":1}
W20260811 18:41:33.282316 16179 heartbeater.cc:644] Failed to heartbeat to 127.30.185.190:41149 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.185.190:41149: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Aug 11 18:41:33 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:33 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:33.291932 16181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:33.282934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.129:52403)
0811 18:41:33.283156 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:33.283178 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:33.283296 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:33.283628 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:33.283634 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:33.284096 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:33.284309 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:33.284323 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.285640 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.285643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:33.286536 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:33.286547 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.287546 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.287550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:33.287685 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:33.288302 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:33.288320 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:33.291549 (+ 3229us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 18:41:33.291985 16183 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:33.283177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:52403 (local address 127.30.185.190:41149)
0811 18:41:33.283353 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:33.283357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:33.283373 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:41:33.283419 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:33.283422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:33.283475 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:33.283553 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:33.284461 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.285493 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.286681 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.287385 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.287547 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:33.291802 (+ 4255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.129:52403: BlockingRecv error: recv got EOF from 127.30.185.129:52403 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 18:41:33 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:33 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:33.302474 16181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:33.292892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.129:53397)
0811 18:41:33.293082 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:33.293111 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:33.293268 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:33.293642 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:33.293646 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:33.293954 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:33.294109 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:33.294117 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.295653 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.295657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:33.296800 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:33.296814 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.297748 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.297754 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:33.297941 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:33.298617 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:33.298636 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:33.302065 (+ 3429us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 18:41:33.302528 16183 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:33.293010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:53397 (local address 127.30.185.190:41149)
0811 18:41:33.293231 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:33.293238 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:33.293260 (+ 22us) server_negotiation.cc:407] Connection header received
0811 18:41:33.293383 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:33.293390 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:33.293487 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:33.293600 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:33.294258 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.295497 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.296963 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.297567 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.297718 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:33.302304 (+ 4586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.129:53397: BlockingRecv error: recv got EOF from 127.30.185.129:53397 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 18:41:33.302970 16179 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:41:33.621985 16185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:33.622285 16185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:33.622464 16185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:33.629890 16185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:33.630033 16185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:33.630220 16185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:41:33.640424 16185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:33105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.130.keytab
--principal=oryx/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:37627
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=43751
--tserver_master_addrs=127.30.185.190:41149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:33.641882 16185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:33.643308 16185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:33.656948 16196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:33.657001 16199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:33.657040 16197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:33.658667 16185 server_base.cc:1034] running on GCE node
I20260811 18:41:33.659344 16185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:33.660378 16185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:33.661580 16185 hybrid_clock.cc:648] HybridClock initialized: now 1786473693661516 us; error 56 us; skew 500 ppm
I20260811 18:41:33.661918 16185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:33 dist-test-slave-z8x9 krb5kdc[14722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473693, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:33.665390 16185 init.cc:377] Logged in from keytab as oryx/127.30.185.130@KRBTEST.COM (short username oryx)
I20260811 18:41:33.666944 16185 webserver.cc:460] Webserver started at http://127.30.185.130:43751/ using document root <none> and password file <none>
I20260811 18:41:33.667701 16185 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:33.667870 16185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:33.674118 16185 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:41:33.677134 16207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:33.678701 16185 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:41:33.678925 16185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "58f60e97098943abbdde5a8e8f3fc3a0"
format_stamp: "Formatted at 2026-08-11 18:41:30 on dist-test-slave-z8x9"
server_key: "59afaad63820d88237d3caf2d36a9b98"
server_key_iv: "c7c67ac38cd88fdb817d71ba3d776549"
server_key_version: "encryptionkey@0"
I20260811 18:41:33.680081 16185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:33.711640 16185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:33.716681 16185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:33.717023 16185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:33.718171 16185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:33.719715 16185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:33.719856 16185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:33.720002 16185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:33.720114 16185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:33.758466 16185 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:37627
I20260811 18:41:33.758576 16438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:37627 every 8 connection(s)
I20260811 18:41:33.760129 16185 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:41:33.765630 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 16185
I20260811 18:41:33.766250 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:33601
--local_ip_for_outbound_sockets=127.30.185.131
--tserver_master_addrs=127.30.185.190:41149
--webserver_port=44121
--webserver_interface=127.30.185.131
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:33105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.185.131
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.131.keytab with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:41:33 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:33 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 18:41:33.779381 16183 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:33.762830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:50639 (local address 127.30.185.190:41149)
0811 18:41:33.763048 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:33.763055 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:33.764406 (+ 1351us) server_negotiation.cc:407] Connection header received
0811 18:41:33.766575 (+ 2169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:33.766580 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:33.766656 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:33.766730 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:33.768555 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.769727 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.771482 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.772210 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.772360 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:33.779055 (+ 6695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.130:50639: BlockingRecv error: recv got EOF from 127.30.185.130:50639 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 18:41:33.780229 16441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:33.763366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.130:50639)
0811 18:41:33.764216 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:33.764269 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:33.766227 (+ 1958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:33.767152 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:33.767160 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:33.767736 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:33.768352 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:33.768371 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.769928 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.769933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:33.771323 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:33.771337 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.772453 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.772459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:33.772689 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:33.773580 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:33.773606 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:33.778816 (+ 5210us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":300,"threads_started":1}
W20260811 18:41:33.781113 16439 heartbeater.cc:644] Failed to heartbeat to 127.30.185.190:41149 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.185.190:41149: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Aug 11 18:41:33 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:33 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:33.792351 16441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:33.781904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.130:50873)
0811 18:41:33.782112 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:33.782133 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:33.782249 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:33.782608 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:33.782613 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:33.783096 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:33.783286 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:33.783299 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.784990 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.784995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:33.786651 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:33.786680 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.787844 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.787851 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:33.788053 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:33.788800 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:33.788824 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:33.792127 (+ 3303us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 18:41:33.792757 16183 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:33.782064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:50873 (local address 127.30.185.190:41149)
0811 18:41:33.782253 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:33.782261 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:33.782285 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:41:33.782349 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:33.782356 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:33.782442 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:33.782562 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:33.783457 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.784779 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.786837 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.787666 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.787816 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:33.792442 (+ 4626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.130:50873: BlockingRecv error: recv got EOF from 127.30.185.130:50873 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 18:41:33 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:33 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:33.802078 16441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:33.793142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.130:45759)
0811 18:41:33.793345 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:33.793366 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:33.793489 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:33.794108 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:33.794113 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:33.794470 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:33.794623 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:33.794633 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.795813 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.795819 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:33.796815 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:33.796827 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.797681 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.797688 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:33.797895 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:33.798586 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:33.798604 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:33.801766 (+ 3162us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":81}
W20260811 18:41:33.802438 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:33.793234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:45759 (local address 127.30.185.190:41149)
0811 18:41:33.793814 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:33.793818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:33.793834 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:41:33.793877 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:33.793881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:33.793948 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:33.794078 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:33.794801 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.795655 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.796963 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:33.797502 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:33.797645 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:33.802020 (+ 4375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.130:45759: BlockingRecv error: recv got EOF from 127.30.185.130:45759 (error 108)
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":299,"threads_started":1}
W20260811 18:41:33.802527 16439 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:41:34.120011 16442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:34.120333 16442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:34.120497 16442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:34.127978 16442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:34.128099 16442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:34.128276 16442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:41:34.138651 16442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:33105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/krb5kdc/oryx_127.30.185.131.keytab
--principal=oryx/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:33601
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=44121
--tserver_master_addrs=127.30.185.190:41149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:34.140050 16442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:34.141412 16442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:34.155134 16452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:34.155088 16456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:34.155050 16454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:34.155925 16442 server_base.cc:1034] running on GCE node
I20260811 18:41:34.156560 16442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:34.160178 16442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:34.161429 16442 hybrid_clock.cc:648] HybridClock initialized: now 1786473694161377 us; error 53 us; skew 500 ppm
I20260811 18:41:34.161749 16442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:34 dist-test-slave-z8x9 krb5kdc[14722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473694, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:34.165153 16442 init.cc:377] Logged in from keytab as oryx/127.30.185.131@KRBTEST.COM (short username oryx)
I20260811 18:41:34.166770 16442 webserver.cc:460] Webserver started at http://127.30.185.131:44121/ using document root <none> and password file <none>
I20260811 18:41:34.167514 16442 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:34.167675 16442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:34.173728 16442 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 18:41:34.176725 16466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:34.178365 16442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:41:34.178567 16442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "d3a3afca517c4cb784cea6a8aa5275e8"
format_stamp: "Formatted at 2026-08-11 18:41:31 on dist-test-slave-z8x9"
server_key: "37f4ccf81ed39e27cbac2f198865ac6a"
server_key_iv: "7cda6a920063a5e5c5b11ae934146c03"
server_key_version: "encryptionkey@0"
I20260811 18:41:34.179585 16442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:34.203729 16442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:34.208748 16442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:34.209050 16442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:34.210215 16442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:34.211593 16442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:34.211696 16442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:34.211864 16442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:34.211966 16442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:34.247512 16442 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:33601
I20260811 18:41:34.247611 16665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:33601 every 8 connection(s)
I20260811 18:41:34.249104 16442 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:41:34.252022 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 16442
Aug 11 18:41:34 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:34 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 18:41:34.269466 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:34.251959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:44245 (local address 127.30.185.190:41149)
0811 18:41:34.252185 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:34.252193 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:34.253657 (+ 1464us) server_negotiation.cc:407] Connection header received
0811 18:41:34.256539 (+ 2882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:34.256547 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:34.256647 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:34.256773 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:34.258884 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:34.259874 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:34.262283 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:34.262790 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:34.262932 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:34.269197 (+ 6265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.131:44245: BlockingRecv error: recv got EOF from 127.30.185.131:44245 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 18:41:34.270038 16670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:34.252480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.131:44245)
0811 18:41:34.253472 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:34.253530 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:34.256192 (+ 2662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:34.257045 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:34.257053 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:34.257815 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:34.258677 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:34.258705 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:34.260117 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:34.260123 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:34.262129 (+ 2006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:34.262144 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:34.263045 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:34.263053 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:34.263293 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:34.264062 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:34.264093 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:34.268958 (+ 4865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":363,"threads_started":1}
W20260811 18:41:34.270735 16666 heartbeater.cc:644] Failed to heartbeat to 127.30.185.190:41149 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.185.190:41149: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
I20260811 18:41:34.271775 16183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:34.253119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53704 (local address 127.30.185.190:41149)
0811 18:41:34.253316 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:34.253322 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:34.253902 (+ 580us) server_negotiation.cc:407] Connection header received
0811 18:41:34.254119 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:34.254123 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:34.254237 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:34.254331 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:34.256496 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:34.257484 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:34.259239 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:34.259891 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:34.260013 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:34.263852 (+ 3839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:34.263873 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:34.263896 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:34.263954 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:34.267403 (+ 3449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:34.268282 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:34.268288 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:34.268295 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:34.268474 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:34.268913 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:34.268917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:34.268921 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:34.269654 (+ 733us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:34.270025 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:34.270949 (+ 924us) server_negotiation.cc:299] Negotiation successful
0811 18:41:34.271308 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":91}
Aug 11 18:41:34 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:34 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:34.279558 16670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:34.271300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.131:53299)
0811 18:41:34.271525 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:34.271545 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:34.271651 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:34.272151 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:34.272156 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:34.272488 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:34.272647 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:34.272655 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:34.273794 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:34.273805 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:34.274886 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:34.274899 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:34.275696 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:34.275702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:34.275879 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:34.276382 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:34.276398 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:34.279345 (+ 2947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 18:41:34.280022 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:34.271466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:53299 (local address 127.30.185.190:41149)
0811 18:41:34.271633 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:34.271637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:34.271653 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:41:34.271786 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:34.271790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:34.271872 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:34.271976 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:34.272792 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:34.273635 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:34.275033 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:34.275544 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:34.275679 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:34.279524 (+ 3845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.131:53299: BlockingRecv error: recv got EOF from 127.30.185.131:53299 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 11 18:41:34 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:34 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:34.288825 16670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:34.280298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.131:34077)
0811 18:41:34.280492 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:34.280513 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:34.280620 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:34.280972 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:34.280977 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:34.281335 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:34.281479 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:34.281487 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:34.282435 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:34.282441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:34.283400 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:34.283433 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:34.284265 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:34.284271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:34.284443 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:34.285083 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:34.285100 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:34.288525 (+ 3425us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 18:41:34.288985 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:34.280332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:34077 (local address 127.30.185.190:41149)
0811 18:41:34.280615 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:34.280621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:34.280638 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:41:34.280730 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:34.280735 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:34.280839 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:34.280946 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:34.281615 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:34.282296 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:34.283567 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:34.284120 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:34.284246 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:34.288730 (+ 4484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.131:34077: BlockingRecv error: recv got EOF from 127.30.185.131:34077 (error 108)
Metrics: {"server-negotiator.queue_time_us":148}
W20260811 18:41:34.289342 16666 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 18:41:34 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:34 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:34.312990 16677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:34.303582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.129:41951)
0811 18:41:34.304093 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:34.304109 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:34.304272 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:34.304627 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:34.304632 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:34.305144 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:34.305400 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:34.305410 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:34.306521 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:34.306532 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:34.307587 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:34.307600 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:34.308378 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:34.308384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:34.308554 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:34.309265 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:34.309283 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:34.312755 (+ 3472us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":269,"threads_started":1}
W20260811 18:41:34.313376 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:34.303662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:41951 (local address 127.30.185.190:41149)
0811 18:41:34.303901 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:34.303910 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:34.304197 (+ 287us) server_negotiation.cc:407] Connection header received
0811 18:41:34.304420 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:34.304430 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:34.304490 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:34.304595 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:34.305547 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:34.306351 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:34.307738 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:34.308179 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:34.308314 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:34.312966 (+ 4652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.129:41951: BlockingRecv error: recv got EOF from 127.30.185.129:41951 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 11 18:41:34 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:34 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:34.811970 16678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:34.803156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.130:50795)
0811 18:41:34.803658 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:34.803675 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:34.803799 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:34.804168 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:34.804171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:34.804661 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:34.804862 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:34.804872 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:34.805985 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:34.805989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:34.807034 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:34.807047 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:34.807753 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:34.807757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:34.807873 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:34.808458 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:34.808475 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:34.811628 (+ 3153us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":281,"threads_started":1}
W20260811 18:41:34.812213 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:34.803210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:50795 (local address 127.30.185.190:41149)
0811 18:41:34.803444 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:34.803454 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:34.803783 (+ 329us) server_negotiation.cc:407] Connection header received
0811 18:41:34.803957 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:34.803962 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:34.804040 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:34.804124 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:34.804992 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:34.805851 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:34.807196 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:34.807599 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:34.807694 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:34.811942 (+ 4248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.130:50795: BlockingRecv error: recv got EOF from 127.30.185.130:50795 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 18:41:35 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:35 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:35.298671 16679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:35.290098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.131:55423)
0811 18:41:35.290677 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:35.290694 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:35.290814 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:35.291143 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:35.291146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:35.291624 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:35.291824 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:35.291834 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:35.292885 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:35.292889 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:35.293972 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:35.293985 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:35.294662 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:35.294666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:35.294779 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:35.295419 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:35.295436 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:35.298433 (+ 2997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":314,"threads_started":1}
W20260811 18:41:35.298918 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:35.290157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:55423 (local address 127.30.185.190:41149)
0811 18:41:35.290344 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:35.290349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:35.290777 (+ 428us) server_negotiation.cc:407] Connection header received
0811 18:41:35.290941 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:35.290944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:35.291009 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:35.291088 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:35.291998 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:35.292747 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:35.294102 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:35.294508 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:35.294601 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:35.298654 (+ 4053us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.131:55423: BlockingRecv error: recv got EOF from 127.30.185.131:55423 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 18:41:35 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:35 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:35.322221 16680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:35.314061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.129:46077)
0811 18:41:35.314594 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:35.314612 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:35.314728 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:35.315023 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:35.315026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:35.315448 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:35.315627 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:35.315636 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:35.316621 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:35.316626 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:35.317624 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:35.317638 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:35.318351 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:35.318356 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:35.318486 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:35.319072 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:35.319091 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:35.321968 (+ 2877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":300,"threads_started":1}
W20260811 18:41:35.322414 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:35.314109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:46077 (local address 127.30.185.190:41149)
0811 18:41:35.314356 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:35.314366 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:35.314766 (+ 400us) server_negotiation.cc:407] Connection header received
0811 18:41:35.314825 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:35.314829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:35.314893 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:35.314980 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:35.315777 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:35.316495 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:35.317766 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:35.318211 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:35.318316 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:35.322179 (+ 3863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.129:46077: BlockingRecv error: recv got EOF from 127.30.185.129:46077 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 18:41:35 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:35 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:35.821632 16682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:35.813111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.130:52731)
0811 18:41:35.813693 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:35.813709 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:35.813848 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:35.814297 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:35.814304 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:35.814877 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:35.815052 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:35.815061 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:35.816145 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:35.816149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:35.817035 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:35.817046 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:35.817760 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:35.817764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:35.817893 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:35.818493 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:35.818510 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:35.821394 (+ 2884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":451,"threads_started":1}
W20260811 18:41:35.821854 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:35.813382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:52731 (local address 127.30.185.190:41149)
0811 18:41:35.813547 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:35.813573 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:35.813791 (+ 218us) server_negotiation.cc:407] Connection header received
0811 18:41:35.814007 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:35.814010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:35.814108 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:35.814223 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:35.815191 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:35.816010 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:35.817211 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:35.817613 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:35.817717 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:35.821605 (+ 3888us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.130:52731: BlockingRecv error: recv got EOF from 127.30.185.130:52731 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 18:41:36 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:36 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:36.309309 16683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:36.299808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.131:52855)
0811 18:41:36.300449 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:36.300467 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:36.300590 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:36.300946 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:36.300949 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:36.301405 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:36.301578 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:36.301586 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:36.302623 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:36.302627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:36.303538 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:36.303550 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:36.304583 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:36.304592 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:36.304816 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:36.305617 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:36.305644 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:36.308930 (+ 3286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":405,"threads_started":1}
W20260811 18:41:36.309584 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:36.300050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:52855 (local address 127.30.185.190:41149)
0811 18:41:36.300256 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:36.300261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:36.300561 (+ 300us) server_negotiation.cc:407] Connection header received
0811 18:41:36.300701 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:36.300705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:36.300771 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:36.300865 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:36.301724 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:36.302479 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:36.303708 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:36.304354 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:36.304510 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:36.309198 (+ 4688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.131:52855: BlockingRecv error: recv got EOF from 127.30.185.131:52855 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 18:41:36 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:36 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:36.332233 16684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:36.323176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.129:45531)
0811 18:41:36.323711 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:36.323728 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:36.323859 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:36.324176 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:36.324179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:36.324538 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:36.324708 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:36.324717 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:36.325808 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:36.325812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:36.326723 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:36.326734 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:36.327419 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:36.327423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:36.327561 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:36.328253 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:36.328272 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:36.331942 (+ 3670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":311,"threads_started":1}
W20260811 18:41:36.332412 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:36.323289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:45531 (local address 127.30.185.190:41149)
0811 18:41:36.323461 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:36.323465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:36.323808 (+ 343us) server_negotiation.cc:407] Connection header received
0811 18:41:36.323985 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:36.323988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:36.324052 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:36.324131 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:36.324835 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:36.325651 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:36.326859 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:36.327267 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:36.327379 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:36.332170 (+ 4791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.129:45531: BlockingRecv error: recv got EOF from 127.30.185.129:45531 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 18:41:36 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:36 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:36.832676 16685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:36.822657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.130:42939)
0811 18:41:36.823233 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:36.823250 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:36.823433 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:36.823859 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:36.823864 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:36.824318 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:36.824506 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:36.824517 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:36.825813 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:36.825821 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:36.826784 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:36.826796 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:36.827578 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:36.827585 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:36.827835 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:36.828725 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:36.828749 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:36.832416 (+ 3667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":311,"threads_started":1}
W20260811 18:41:36.832996 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:36.822799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:42939 (local address 127.30.185.190:41149)
0811 18:41:36.823059 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:36.823067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:36.823347 (+ 280us) server_negotiation.cc:407] Connection header received
0811 18:41:36.823604 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:36.823612 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:36.823703 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:36.823822 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:36.824670 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:36.825664 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:36.826937 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:36.827387 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:36.827573 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:36.832740 (+ 5167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.130:42939: BlockingRecv error: recv got EOF from 127.30.185.130:42939 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 11 18:41:37 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:37 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:37.319346 16686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:37.310334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.131:50537)
0811 18:41:37.310877 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:37.310894 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:37.311014 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:37.311312 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:37.311315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:37.311658 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:37.311841 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:37.311850 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:37.312883 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:37.312887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:37.313836 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:37.313848 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:37.314524 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:37.314528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:37.314686 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:37.315425 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:37.315454 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:37.318950 (+ 3496us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":313,"threads_started":1}
W20260811 18:41:37.319442 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:37.310480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:50537 (local address 127.30.185.190:41149)
0811 18:41:37.310771 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:37.310777 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:37.310981 (+ 204us) server_negotiation.cc:407] Connection header received
0811 18:41:37.311115 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:37.311119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:37.311184 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:37.311294 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:37.311971 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:37.312761 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:37.313966 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:37.314393 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:37.314486 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:37.319176 (+ 4690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.131:50537: BlockingRecv error: recv got EOF from 127.30.185.131:50537 (error 108)
Metrics: {"server-negotiator.queue_time_us":136}
Aug 11 18:41:37 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:37 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:37.342315 16689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:37.333366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.129:39531)
0811 18:41:37.333869 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:37.333885 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:37.334010 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:37.334306 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:37.334310 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:37.334667 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:37.334843 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:37.334852 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:37.335888 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:37.335892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:37.336966 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:37.336981 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:37.337714 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:37.337719 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:37.337843 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:37.338498 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:37.338517 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:37.341920 (+ 3403us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":275,"threads_started":1}
W20260811 18:41:37.342401 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:37.333416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:39531 (local address 127.30.185.190:41149)
0811 18:41:37.333667 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:37.333674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:37.333966 (+ 292us) server_negotiation.cc:407] Connection header received
0811 18:41:37.334110 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:37.334114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:37.334175 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:37.334266 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:37.334966 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:37.335722 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:37.337096 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:37.337581 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:37.337678 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:37.342133 (+ 4455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.129:39531: BlockingRecv error: recv got EOF from 127.30.185.129:39531 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 11 18:41:37 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:37 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:37.847149 16692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:37.833693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.130:34309)
0811 18:41:37.834238 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:37.834262 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:37.834381 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:37.834704 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:37.834715 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:37.835111 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:37.835271 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:37.835280 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:37.836333 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:37.836338 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:37.837552 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:37.837567 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:37.838369 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:37.838379 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:37.838566 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:37.843368 (+ 4802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:37.843402 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:37.846907 (+ 3505us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":295,"threads_started":1}
W20260811 18:41:37.847437 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:37.833772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:34309 (local address 127.30.185.190:41149)
0811 18:41:37.833951 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:37.833955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:37.834342 (+ 387us) server_negotiation.cc:407] Connection header received
0811 18:41:37.834481 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:37.834485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:37.834570 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:37.834655 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:37.835407 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:37.836159 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:37.837698 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:37.838149 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:37.838249 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:37.847126 (+ 8877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.130:34309: BlockingRecv error: recv got EOF from 127.30.185.130:34309 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 18:41:38 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:38 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:38.329437 16693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:38.320402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.131:59887)
0811 18:41:38.321030 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:38.321047 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:38.321189 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:38.321548 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:38.321551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:38.321921 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:38.322072 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:38.322080 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:38.323070 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:38.323074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:38.324057 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:38.324069 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:38.324860 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:38.324865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:38.325001 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:38.325802 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:38.325820 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:38.329190 (+ 3370us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":381,"threads_started":1}
W20260811 18:41:38.329771 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:38.320533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:59887 (local address 127.30.185.190:41149)
0811 18:41:38.320765 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:38.320771 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:38.321117 (+ 346us) server_negotiation.cc:407] Connection header received
0811 18:41:38.321336 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:38.321340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:38.321403 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:38.321486 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:38.322199 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:38.322953 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:38.324239 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:38.324692 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:38.324818 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:38.329478 (+ 4660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.131:59887: BlockingRecv error: recv got EOF from 127.30.185.131:59887 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 18:41:38 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:38 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:38.353698 16694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:38.343705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.129:53923)
0811 18:41:38.344399 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:38.344421 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:38.344555 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:38.344917 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:38.344922 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:38.345438 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:38.345607 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:38.345617 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:38.346819 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:38.346825 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:38.347908 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:38.347922 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:38.348654 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:38.348660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:38.348860 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:38.349617 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:38.349667 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:38.353421 (+ 3754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":366,"threads_started":1}
W20260811 18:41:38.353940 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:38.343843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:53923 (local address 127.30.185.190:41149)
0811 18:41:38.344112 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:38.344120 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:38.344522 (+ 402us) server_negotiation.cc:407] Connection header received
0811 18:41:38.344682 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:38.344686 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:38.344751 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:38.344880 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:38.345876 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:38.346669 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:38.348062 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:38.348466 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:38.348620 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:38.353658 (+ 5038us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.129:53923: BlockingRecv error: recv got EOF from 127.30.185.129:53923 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 18:41:38 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:38 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:38.857765 16695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:38.848260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.130:38395)
0811 18:41:38.848851 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:38.848867 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:38.849019 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:38.849520 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:38.849525 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:38.850118 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:38.850302 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:38.850312 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:38.851314 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:38.851317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:38.852240 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:38.852251 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:38.852932 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:38.852936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:38.853067 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:38.853797 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:38.853815 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:38.857481 (+ 3666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":298,"threads_started":1}
W20260811 18:41:38.858064 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:38.848507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:38395 (local address 127.30.185.190:41149)
0811 18:41:38.848721 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:38.848730 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:38.848953 (+ 223us) server_negotiation.cc:407] Connection header received
0811 18:41:38.849145 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:38.849150 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:38.849296 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:38.849406 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:38.850445 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:38.851182 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:38.852387 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:38.852800 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:38.852903 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:38.857757 (+ 4854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.130:38395: BlockingRecv error: recv got EOF from 127.30.185.130:38395 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 11 18:41:39 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:39 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:39.339166 16696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:39.330453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.131:33547)
0811 18:41:39.331028 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:39.331045 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:39.331156 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:39.331535 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:39.331542 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:39.332074 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:39.332282 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:39.332291 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:39.333315 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:39.333320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:39.334236 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:39.334260 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:39.335014 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:39.335018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:39.335144 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:39.335847 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:39.335873 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:39.338904 (+ 3031us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":295,"threads_started":1}
W20260811 18:41:39.339470 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:39.330534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:33547 (local address 127.30.185.190:41149)
0811 18:41:39.330757 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:39.330764 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:39.331150 (+ 386us) server_negotiation.cc:407] Connection header received
0811 18:41:39.331275 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:39.331280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:39.331383 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:39.331466 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:39.332416 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:39.333111 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:39.334396 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:39.334881 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:39.334981 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:39.339162 (+ 4181us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.131:33547: BlockingRecv error: recv got EOF from 127.30.185.131:33547 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 18:41:39 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:39 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:39.363708 16697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:39.354891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.129:42079)
0811 18:41:39.355458 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:39.355481 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:39.355625 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:39.356068 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:39.356073 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:39.356581 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:39.356737 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:39.356746 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:39.357900 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:39.357905 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:39.358829 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:39.358841 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:39.359604 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:39.359610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:39.359795 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:39.360445 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:39.360464 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:39.363480 (+ 3016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":886,"threads_started":1}
W20260811 18:41:39.363991 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:39.355339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:42079 (local address 127.30.185.190:41149)
0811 18:41:39.355531 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:39.355539 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:39.355559 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:41:39.355798 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:39.355805 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:39.355918 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:39.356041 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:39.356895 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:39.357738 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:39.358980 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:39.359439 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:39.359575 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:39.363692 (+ 4117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.129:42079: BlockingRecv error: recv got EOF from 127.30.185.129:42079 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 11 18:41:39 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:39 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:39.867616 16700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:39.858886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.130:42535)
0811 18:41:39.859466 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:39.859483 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:39.859604 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:39.859934 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:39.859937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:39.860328 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:39.860499 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:39.860508 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:39.861656 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:39.861660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:39.862690 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:39.862703 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:39.863401 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:39.863406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:39.863570 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:39.864335 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:39.864353 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:39.867360 (+ 3007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":320,"threads_started":1}
W20260811 18:41:39.867926 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:39.858923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:42535 (local address 127.30.185.190:41149)
0811 18:41:39.859111 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:39.859116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:39.859598 (+ 482us) server_negotiation.cc:407] Connection header received
0811 18:41:39.859723 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:39.859726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:39.859791 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:39.859881 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:39.860668 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:39.861507 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:39.862831 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:39.863232 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:39.863335 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:39.867608 (+ 4273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.130:42535: BlockingRecv error: recv got EOF from 127.30.185.130:42535 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 18:41:40 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:40 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:40.350405 16702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:40.340320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.131:42217)
0811 18:41:40.340922 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:40.340939 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:40.341066 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:40.341525 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:40.341530 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:40.342096 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:40.342334 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:40.342352 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:40.343773 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:40.343778 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:40.345220 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:40.345240 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:40.346084 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:40.346088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:40.346260 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:40.346975 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:40.346999 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:40.350023 (+ 3024us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":337,"threads_started":1}
W20260811 18:41:40.350518 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:40.340399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:42217 (local address 127.30.185.190:41149)
0811 18:41:40.340582 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:40.340587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:40.341022 (+ 435us) server_negotiation.cc:407] Connection header received
0811 18:41:40.341272 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:40.341277 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:40.341345 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:40.341487 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:40.342555 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:40.343581 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:40.345393 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:40.345909 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:40.346013 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:40.350258 (+ 4245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.131:42217: BlockingRecv error: recv got EOF from 127.30.185.131:42217 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 18:41:40 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:40 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:40.373908 16704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:40.364719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.129:52039)
0811 18:41:40.365402 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:40.365420 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:40.365585 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:40.365947 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:40.365952 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:40.366436 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:40.366607 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:40.366616 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:40.367695 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:40.367703 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:40.368813 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:40.368827 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:40.369825 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:40.369831 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:40.370003 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:40.370679 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:40.370699 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:40.373677 (+ 2978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":268,"threads_started":1}
W20260811 18:41:40.374259 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:40.364830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:52039 (local address 127.30.185.190:41149)
0811 18:41:40.365052 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:40.365061 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:40.365511 (+ 450us) server_negotiation.cc:407] Connection header received
0811 18:41:40.365725 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:40.365729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:40.365791 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:40.365898 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:40.366761 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:40.367544 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:40.368957 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:40.369612 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:40.369799 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:40.373876 (+ 4077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.129:52039: BlockingRecv error: recv got EOF from 127.30.185.129:52039 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 18:41:40 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:40 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:40.877470 16705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:40.868692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.130:41699)
0811 18:41:40.869374 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:40.869390 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:40.869562 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:40.869920 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:40.869925 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:40.870399 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:40.870571 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:40.870579 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:40.871692 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:40.871696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:40.872623 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:40.872634 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:40.873415 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:40.873421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:40.873601 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:40.874165 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:40.874199 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:40.877153 (+ 2954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":273,"threads_started":1}
W20260811 18:41:40.877625 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:40.868748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:41699 (local address 127.30.185.190:41149)
0811 18:41:40.869012 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:40.869021 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:40.869484 (+ 463us) server_negotiation.cc:407] Connection header received
0811 18:41:40.869693 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:40.869698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:40.869777 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:40.869883 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:40.870722 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:40.871480 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:40.872773 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:40.873260 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:40.873399 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:40.877410 (+ 4011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.130:41699: BlockingRecv error: recv got EOF from 127.30.185.130:41699 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
Aug 11 18:41:41 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:41 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:41.360538 16708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:41.351470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.131:35001)
0811 18:41:41.352115 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:41.352132 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:41.352252 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:41.352559 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:41.352563 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:41.352997 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:41.353231 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:41.353242 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:41.354269 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:41.354272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:41.355177 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:41.355188 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:41.355910 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:41.355915 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:41.356099 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:41.356892 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:41.356920 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:41.360177 (+ 3257us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":308,"threads_started":1}
W20260811 18:41:41.360864 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:41.351559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:35001 (local address 127.30.185.190:41149)
0811 18:41:41.351743 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:41.351748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:41.352210 (+ 462us) server_negotiation.cc:407] Connection header received
0811 18:41:41.352351 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:41.352355 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:41.352417 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:41.352500 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:41.353369 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:41.354138 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:41.355329 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:41.355735 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:41.355879 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:41.360576 (+ 4697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.131:35001: BlockingRecv error: recv got EOF from 127.30.185.131:35001 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 18:41:41 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:41 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:41.383764 16709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:41.374824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.129:58629)
0811 18:41:41.375398 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:41.375415 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:41.375565 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:41.375930 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:41.375935 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:41.376440 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:41.376612 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:41.376621 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:41.377707 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:41.377713 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:41.378618 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:41.378630 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:41.379441 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:41.379449 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:41.379607 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:41.380281 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:41.380299 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:41.383377 (+ 3078us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":294,"threads_started":1}
W20260811 18:41:41.383912 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:41.375021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:58629 (local address 127.30.185.190:41149)
0811 18:41:41.375254 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:41.375260 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:41.375495 (+ 235us) server_negotiation.cc:407] Connection header received
0811 18:41:41.375710 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:41.375714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:41.375791 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:41.375895 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:41.376757 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:41.377565 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:41.378773 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:41.379237 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:41.379377 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:41.383644 (+ 4267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.129:58629: BlockingRecv error: recv got EOF from 127.30.185.129:58629 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 18:41:41 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:41 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:41.887163 16710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:41.878394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.130:56687)
0811 18:41:41.878912 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:41.878929 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:41.879044 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:41.879331 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:41.879335 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:41.879721 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:41.879875 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:41.879883 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:41.881021 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:41.881025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:41.882030 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:41.882044 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:41.882744 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:41.882749 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:41.882866 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:41.883533 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:41.883550 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:41.886931 (+ 3381us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":280,"threads_started":1}
W20260811 18:41:41.887463 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:41.878509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:56687 (local address 127.30.185.190:41149)
0811 18:41:41.878753 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:41.878760 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:41.879008 (+ 248us) server_negotiation.cc:407] Connection header received
0811 18:41:41.879140 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:41.879143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:41.879203 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:41.879284 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:41.879997 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:41.880897 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:41.882157 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:41.882577 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:41.882677 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:41.887170 (+ 4493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.130:56687: BlockingRecv error: recv got EOF from 127.30.185.130:56687 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 11 18:41:42 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:42 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:42.371457 16711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:42.361679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.131:58223)
0811 18:41:42.362352 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:42.362375 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:42.362536 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:42.363160 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:42.363167 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:42.363764 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:42.363980 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:42.363993 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:42.365452 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:42.365458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:42.366908 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:42.366922 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:42.367596 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:42.367600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:42.367710 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:42.368383 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:42.368402 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:42.371205 (+ 2803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":325,"threads_started":1}
W20260811 18:41:42.371654 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:42.361800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:58223 (local address 127.30.185.190:41149)
0811 18:41:42.362057 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:42.362063 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:42.362465 (+ 402us) server_negotiation.cc:407] Connection header received
0811 18:41:42.362713 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:42.362722 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:42.362806 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:42.362961 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:42.364159 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:42.365298 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:42.367047 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:42.367438 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:42.367536 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:42.371415 (+ 3879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.131:58223: BlockingRecv error: recv got EOF from 127.30.185.131:58223 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 11 18:41:42 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:42 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:42.394232 16713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:42.384903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.129:43549)
0811 18:41:42.385484 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:42.385502 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:42.385640 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:42.385975 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:42.385979 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:42.386453 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:42.386699 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:42.386712 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:42.388223 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:42.388230 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:42.389325 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:42.389366 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:42.390125 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:42.390129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:42.390269 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:42.390912 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:42.390929 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:42.393973 (+ 3044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":356,"threads_started":1}
W20260811 18:41:42.394519 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:42.384993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:43549 (local address 127.30.185.190:41149)
0811 18:41:42.385264 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:42.385271 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:42.385592 (+ 321us) server_negotiation.cc:407] Connection header received
0811 18:41:42.385768 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:42.385772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:42.385839 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:42.385925 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:42.386852 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:42.388074 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:42.389555 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:42.389976 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:42.390075 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:42.394201 (+ 4126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.129:43549: BlockingRecv error: recv got EOF from 127.30.185.129:43549 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 18:41:42 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:42 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:42.900856 16714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:42.888191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.130:44259)
0811 18:41:42.888783 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:42.888800 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:42.888983 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:42.889453 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:42.889459 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:42.889912 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:42.890080 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:42.890090 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:42.891120 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:42.891126 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:42.892097 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:42.892109 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:42.892889 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:42.892896 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:42.893074 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:42.897447 (+ 4373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:42.897466 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:42.900600 (+ 3134us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":319,"threads_started":1}
W20260811 18:41:42.901122 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:42.888334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:44259 (local address 127.30.185.190:41149)
0811 18:41:42.888598 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:42.888608 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:42.888902 (+ 294us) server_negotiation.cc:407] Connection header received
0811 18:41:42.889149 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:42.889153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:42.889281 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:42.889391 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:42.890234 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:42.890970 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:42.892255 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:42.892711 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:42.892853 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:42.900840 (+ 7987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.130:44259: BlockingRecv error: recv got EOF from 127.30.185.130:44259 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 18:41:43 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:43 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:43.386161 16715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:43.372701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.131:38331)
0811 18:41:43.377578 (+ 4877us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:43.377595 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:43.377718 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:43.378009 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:43.378013 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:43.378415 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:43.378579 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:43.378588 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:43.379586 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:43.379589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:43.380568 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:43.380581 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:43.381430 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:43.381542 (+ 112us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:43.381755 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:43.382332 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:43.382350 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:43.385910 (+ 3560us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":4777,"thread_start_us":4625,"threads_started":1}
W20260811 18:41:43.386442 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:43.372790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:38331 (local address 127.30.185.190:41149)
0811 18:41:43.372985 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:43.372992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:43.377687 (+ 4695us) server_negotiation.cc:407] Connection header received
0811 18:41:43.377805 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:43.377809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:43.377873 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:43.377956 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:43.378707 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:43.379459 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:43.380698 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:43.381155 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:43.381351 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:43.386136 (+ 4785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.131:38331: BlockingRecv error: recv got EOF from 127.30.185.131:38331 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 11 18:41:43 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:43 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:43.404148 16716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:43.395284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.129:48153)
0811 18:41:43.395846 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:43.395865 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:43.395983 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:43.396390 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:43.396394 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:43.396922 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:43.397082 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:43.397092 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:43.398585 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:43.398589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:43.399532 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:43.399544 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:43.400335 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:43.400340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:43.400466 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:43.401064 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:43.401081 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:43.403915 (+ 2834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":321,"threads_started":1}
W20260811 18:41:43.404388 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:43.395388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:48153 (local address 127.30.185.190:41149)
0811 18:41:43.395581 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:43.395586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:43.395961 (+ 375us) server_negotiation.cc:407] Connection header received
0811 18:41:43.396105 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:43.396109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:43.396196 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:43.396332 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:43.397286 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:43.398424 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:43.399682 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:43.400191 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:43.400299 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:43.404100 (+ 3801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.129:48153: BlockingRecv error: recv got EOF from 127.30.185.129:48153 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 18:41:43 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:43 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:43.910820 16717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:43.901872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.130:42251)
0811 18:41:43.902428 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:43.902445 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:43.902599 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:43.902979 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:43.902983 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:43.903422 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:43.903579 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:43.903588 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:43.904668 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:43.904674 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:43.905632 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:43.905644 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:43.906429 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:43.906436 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:43.906609 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:43.907307 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:43.907324 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:43.910494 (+ 3170us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":299,"threads_started":1}
W20260811 18:41:43.911017 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:43.901992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:42251 (local address 127.30.185.190:41149)
0811 18:41:43.902223 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:43.902232 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:43.902535 (+ 303us) server_negotiation.cc:407] Connection header received
0811 18:41:43.902729 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:43.902734 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:43.902831 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:43.902942 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:43.903732 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:43.904513 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:43.905799 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:43.906263 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:43.906395 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:43.910720 (+ 4325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.130:42251: BlockingRecv error: recv got EOF from 127.30.185.130:42251 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 18:41:44 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:44 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:44.400643 16718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:44.387105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.131:46075)
0811 18:41:44.387685 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:44.387703 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:44.387821 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:44.388091 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:44.388094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:44.388508 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:44.388664 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:44.388673 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:44.389735 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:44.389740 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:44.390703 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:44.390716 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:44.391461 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:44.391466 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:44.391591 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:44.397550 (+ 5959us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:44.397572 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:44.400415 (+ 2843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":332,"threads_started":1}
W20260811 18:41:44.400877 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:44.387239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:46075 (local address 127.30.185.190:41149)
0811 18:41:44.387476 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:44.387480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:44.387774 (+ 294us) server_negotiation.cc:407] Connection header received
0811 18:41:44.387930 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:44.387933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:44.387995 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:44.388076 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:44.388813 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:44.389595 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:44.390832 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:44.391287 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:44.391382 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:44.400629 (+ 9247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.131:46075: BlockingRecv error: recv got EOF from 127.30.185.131:46075 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
Aug 11 18:41:44 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:44 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:44.428155 16721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:44.405268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.129:45061)
0811 18:41:44.416300 (+ 11032us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:44.416328 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:44.416495 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:44.416918 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:44.416926 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:44.417575 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:44.417841 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:44.417854 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:44.419583 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:44.419588 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:44.421391 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:44.421409 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:44.422558 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:44.422563 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:44.422701 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:44.423490 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:44.423519 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:44.427755 (+ 4236us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":10873,"thread_start_us":10748,"threads_started":1}
W20260811 18:41:44.428270 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:44.405366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:45061 (local address 127.30.185.190:41149)
0811 18:41:44.405572 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:44.405579 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:44.416420 (+ 10841us) server_negotiation.cc:407] Connection header received
0811 18:41:44.416641 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:44.416648 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:44.416747 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:44.416855 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:44.418002 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:44.419440 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:44.421598 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:44.422375 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:44.422525 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:44.427990 (+ 5465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.129:45061: BlockingRecv error: recv got EOF from 127.30.185.129:45061 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 18:41:44 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:44 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:44.920426 16722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:44.911857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.130:56127)
0811 18:41:44.912407 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:44.912423 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:44.912539 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:44.912847 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:44.912850 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:44.913373 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:44.913557 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:44.913566 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:44.914542 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:44.914546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:44.915447 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:44.915458 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:44.916148 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:44.916152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:44.916277 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:44.916950 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:44.916968 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:44.920113 (+ 3145us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":305,"threads_started":1}
W20260811 18:41:44.920671 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:44.911986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:56127 (local address 127.30.185.190:41149)
0811 18:41:44.912162 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:44.912166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:44.912499 (+ 333us) server_negotiation.cc:407] Connection header received
0811 18:41:44.912636 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:44.912640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:44.912703 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:44.912789 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:44.913685 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:44.914430 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:44.915573 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:44.916018 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:44.916111 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:44.920419 (+ 4308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.130:56127: BlockingRecv error: recv got EOF from 127.30.185.130:56127 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 18:41:45 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:45 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:45.410171 16725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:45.401597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.131:55891)
0811 18:41:45.402147 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:45.402164 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:45.402308 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:45.402614 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:45.402617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:45.403039 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:45.403221 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:45.403229 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:45.404238 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:45.404242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:45.405259 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:45.405276 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:45.406071 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:45.406075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:45.406203 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:45.406939 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:45.406971 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:45.409920 (+ 2949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":306,"threads_started":1}
W20260811 18:41:45.410393 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:45.401691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:55891 (local address 127.30.185.190:41149)
0811 18:41:45.401870 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:45.401875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:45.402234 (+ 359us) server_negotiation.cc:407] Connection header received
0811 18:41:45.402423 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:45.402427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:45.402487 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:45.402570 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:45.403389 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:45.404119 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:45.405411 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:45.405937 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:45.406033 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:45.410124 (+ 4091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.131:55891: BlockingRecv error: recv got EOF from 127.30.185.131:55891 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 18:41:45 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:45 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:45.438894 16726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:45.429422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.129:42273)
0811 18:41:45.429958 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:45.429974 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:45.430133 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:45.430509 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:45.430516 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:45.430993 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:45.431155 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:45.431164 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:45.432281 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:45.432285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:45.433296 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:45.433309 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:45.434023 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:45.434028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:45.434221 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:45.434909 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:45.434928 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:45.438536 (+ 3608us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":299,"threads_started":1}
W20260811 18:41:45.439026 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:45.429548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:42273 (local address 127.30.185.190:41149)
0811 18:41:45.429781 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:45.429789 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:45.430058 (+ 269us) server_negotiation.cc:407] Connection header received
0811 18:41:45.430257 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:45.430262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:45.430334 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:45.430452 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:45.431301 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:45.432121 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:45.433441 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:45.433871 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:45.434010 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:45.438761 (+ 4751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.129:42273: BlockingRecv error: recv got EOF from 127.30.185.129:42273 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 18:41:45 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:45 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:45.930438 16727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:45.921463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.130:43725)
0811 18:41:45.922032 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:45.922050 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:45.922199 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:45.922520 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:45.922523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:45.922936 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:45.923100 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:45.923109 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:45.924193 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:45.924197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:45.925126 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:45.925138 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:45.926013 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:45.926022 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:45.926183 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:45.927029 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:45.927052 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:45.930200 (+ 3148us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":346,"threads_started":1}
W20260811 18:41:45.930660 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:45.921605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:43725 (local address 127.30.185.190:41149)
0811 18:41:45.921790 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:45.921795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:45.922138 (+ 343us) server_negotiation.cc:407] Connection header received
0811 18:41:45.922311 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:45.922315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:45.922378 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:45.922472 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:45.923231 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:45.924051 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:45.925299 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:45.925848 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:45.925968 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:45.930413 (+ 4445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.130:43725: BlockingRecv error: recv got EOF from 127.30.185.130:43725 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 18:41:46 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:46 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:46.420161 16728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:46.411275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.131:39007)
0811 18:41:46.411778 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:46.411796 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:46.411917 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:46.412302 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:46.412305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:46.412711 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:46.412869 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:46.412878 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:46.413955 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:46.413960 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:46.415006 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:46.415025 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:46.416001 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:46.416006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:46.416188 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:46.416955 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:46.416973 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:46.419929 (+ 2956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":297,"threads_started":1}
W20260811 18:41:46.420423 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:46.411348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:39007 (local address 127.30.185.190:41149)
0811 18:41:46.411606 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:46.411613 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:46.411888 (+ 275us) server_negotiation.cc:407] Connection header received
0811 18:41:46.412081 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:46.412085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:46.412152 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:46.412231 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:46.413010 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:46.413797 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:46.415175 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:46.415827 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:46.415928 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:46.420130 (+ 4202us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.131:39007: BlockingRecv error: recv got EOF from 127.30.185.131:39007 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 11 18:41:46 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:46 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:46.449263 16732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:46.440037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.129:52645)
0811 18:41:46.440660 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:46.440682 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:46.440840 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:46.441379 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:46.441390 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:46.441921 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:46.442083 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:46.442092 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:46.443237 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:46.443241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:46.444284 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:46.444296 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:46.445163 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:46.445181 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:46.445300 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:46.445976 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:46.445994 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:46.448946 (+ 2952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":314,"threads_started":1}
W20260811 18:41:46.449525 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:46.440128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:52645 (local address 127.30.185.190:41149)
0811 18:41:46.440392 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:46.440401 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:46.440777 (+ 376us) server_negotiation.cc:407] Connection header received
0811 18:41:46.441090 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:46.441096 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:46.441224 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:46.441311 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:46.442239 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:46.443104 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:46.444434 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:46.444985 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:46.445102 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:46.449154 (+ 4052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.129:52645: BlockingRecv error: recv got EOF from 127.30.185.129:52645 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 18:41:46 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:46 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:46.940778 16733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:46.931581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.130:35857)
0811 18:41:46.932223 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:46.932241 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:46.932367 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:46.932693 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:46.932697 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:46.933305 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:46.933488 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:46.933498 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:46.934537 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:46.934542 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:46.935565 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:46.935577 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:46.936340 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:46.936346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:46.936475 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:46.937258 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:46.937277 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:46.940350 (+ 3073us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":342,"threads_started":1}
W20260811 18:41:46.940886 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:46.931718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:35857 (local address 127.30.185.190:41149)
0811 18:41:46.931902 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:46.931907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:46.932323 (+ 416us) server_negotiation.cc:407] Connection header received
0811 18:41:46.932458 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:46.932462 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:46.932524 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:46.932607 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:46.933624 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:46.934393 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:46.935701 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:46.936136 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:46.936236 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:46.940586 (+ 4350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.130:35857: BlockingRecv error: recv got EOF from 127.30.185.130:35857 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 18:41:47 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:47 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:47.430003 16735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:47.421233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.131:54247)
0811 18:41:47.421747 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:47.421764 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:47.421883 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:47.422195 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:47.422199 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:47.422634 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:47.422804 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:47.422814 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:47.423764 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:47.423768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:47.424773 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:47.424789 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:47.425910 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:47.425919 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:47.426058 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:47.426786 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:47.426805 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:47.429743 (+ 2938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":271,"threads_started":1}
W20260811 18:41:47.430212 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:47.421285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:54247 (local address 127.30.185.190:41149)
0811 18:41:47.421527 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:47.421536 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:47.421858 (+ 322us) server_negotiation.cc:407] Connection header received
0811 18:41:47.421986 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:47.421989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:47.422068 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:47.422152 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:47.422957 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:47.423641 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:47.424968 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:47.425696 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:47.425831 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:47.429942 (+ 4111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.131:54247: BlockingRecv error: recv got EOF from 127.30.185.131:54247 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 18:41:47 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:47 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:47.458612 16736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:47.450327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.129:55327)
0811 18:41:47.450888 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:47.450905 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:47.451018 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:47.451300 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:47.451303 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:47.451666 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:47.451864 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:47.451874 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:47.452812 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:47.452816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:47.453798 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:47.453811 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:47.454505 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:47.454510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:47.454630 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:47.455327 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:47.455345 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:47.458387 (+ 3042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":291,"threads_started":1}
W20260811 18:41:47.458904 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:47.450391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:55327 (local address 127.30.185.190:41149)
0811 18:41:47.450649 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:47.450653 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:47.450978 (+ 325us) server_negotiation.cc:407] Connection header received
0811 18:41:47.451113 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:47.451116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:47.451176 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:47.451256 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:47.451989 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:47.452667 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:47.453925 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:47.454380 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:47.454473 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:47.458605 (+ 4132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.129:55327: BlockingRecv error: recv got EOF from 127.30.185.129:55327 (error 108)
Metrics: {"server-negotiator.queue_time_us":139}
Aug 11 18:41:47 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:47 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:47.954130 16741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:47.941822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.130:48499)
0811 18:41:47.942382 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:47.942397 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:47.942562 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:47.942963 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:47.942968 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:47.943387 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:47.943570 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:47.943579 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:47.944674 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:47.944683 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:47.945660 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:47.945672 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:47.946428 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:47.946434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:47.946611 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:47.950298 (+ 3687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:47.950316 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:47.953848 (+ 3532us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":294,"threads_started":1}
W20260811 18:41:47.954319 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:47.941971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:48499 (local address 127.30.185.190:41149)
0811 18:41:47.942208 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:47.942216 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:47.942487 (+ 271us) server_negotiation.cc:407] Connection header received
0811 18:41:47.942697 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:47.942709 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:47.942815 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:47.942922 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:47.943722 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:47.944508 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:47.945806 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:47.946274 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:47.946415 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:47.954062 (+ 7647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.130:48499: BlockingRecv error: recv got EOF from 127.30.185.130:48499 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 18:41:48 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:48 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.131@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:48.439743 16742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:48.430997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.131:35393)
0811 18:41:48.431528 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:48.431545 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:48.431705 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:48.432096 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:48.432101 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:48.432521 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:48.432697 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:48.432706 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:48.433844 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:48.433856 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:48.434759 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:48.434770 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:48.435543 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:48.435549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:48.435727 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:48.436405 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:48.436423 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:48.439459 (+ 3036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":286,"threads_started":1}
W20260811 18:41:48.440092 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:48.431120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:35393 (local address 127.30.185.190:41149)
0811 18:41:48.431360 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:48.431368 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:48.431638 (+ 270us) server_negotiation.cc:407] Connection header received
0811 18:41:48.431842 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:48.431848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:48.431935 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:48.432043 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:48.432878 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:48.433694 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:48.434904 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:48.435384 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:48.435521 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:48.439792 (+ 4271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.131:35393: BlockingRecv error: recv got EOF from 127.30.185.131:35393 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 18:41:48 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:48 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.129@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:48.468632 16744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:48.459843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.129:33487)
0811 18:41:48.460424 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:48.460441 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:48.460554 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:48.460871 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:48.460874 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:48.461390 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:48.461565 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:48.461574 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:48.462614 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:48.462618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:48.463553 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:48.463565 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:48.464229 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:48.464235 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:48.464422 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:48.465145 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:48.465169 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:48.468355 (+ 3186us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":294,"threads_started":1}
W20260811 18:41:48.468824 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:48.459980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:33487 (local address 127.30.185.190:41149)
0811 18:41:48.460207 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:48.460212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:48.460524 (+ 312us) server_negotiation.cc:407] Connection header received
0811 18:41:48.460658 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:48.460661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:48.460722 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:48.460810 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:48.461698 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:48.462478 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:48.463682 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:48.464083 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:48.464174 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:48.468566 (+ 4392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.129:33487: BlockingRecv error: recv got EOF from 127.30.185.129:33487 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 11 18:41:48 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:41:48 dist-test-slave-z8x9 krb5kdc[14722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.185.130@KRBTEST.COM for oryx/127.30.185.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:41:48.964272 16746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:48.955561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41149 (local address 127.30.185.130:42781)
0811 18:41:48.956125 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:48.956144 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:48.956267 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:48.956601 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:48.956604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:48.957040 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:48.957293 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:48.957304 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:48.958349 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:48.958354 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:48.959324 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:48.959338 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:48.960122 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:48.960129 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:48.960322 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:48.960908 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:48.960925 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:48.963870 (+ 2945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.185.190:41149: Server oryx/127.30.185.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":314,"threads_started":1}
W20260811 18:41:48.964381 16443 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:48.955627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:42781 (local address 127.30.185.190:41149)
0811 18:41:48.955902 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:48.955910 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:48.956233 (+ 323us) server_negotiation.cc:407] Connection header received
0811 18:41:48.956388 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:48.956392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:48.956457 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:48.956548 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:48.957429 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:48.958181 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:48.959485 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:48.959963 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:48.960104 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:48.964091 (+ 3987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.185.130:42781: BlockingRecv error: recv got EOF from 127.30.185.130:42781 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 18:41:49.262569 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 15886
I20260811 18:41:49.275939 16748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:49.263675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50974 (local address 127.30.185.129:43575)
0811 18:41:49.264358 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:49.264380 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:49.264795 (+ 415us) server_negotiation.cc:407] Connection header received
0811 18:41:49.264995 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:49.265010 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:49.265308 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:49.265512 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:49.266986 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:49.268028 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:49.269305 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:49.269868 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:49.270013 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:49.272583 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:49.272592 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:49.272606 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:49.272643 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:49.275248 (+ 2605us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.30.185.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 18:41:49.275619 (+ 371us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50974: Request ticket server kudu/127.30.185.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":363,"threads_started":1}
W20260811 18:41:49.276080 16748 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50974: Request ticket server kudu/127.30.185.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 18:41:49.276165 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 15886: Not authorized: Client connection negotiation failed: client connection to 127.30.185.129:43575: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.30.185.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 18:41:49.276361 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 15886
I20260811 18:41:49.305223 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 16185
I20260811 18:41:49.317778 16749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:49.306277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57500 (local address 127.30.185.130:37627)
0811 18:41:49.306896 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:49.306904 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:49.306937 (+ 33us) server_negotiation.cc:407] Connection header received
0811 18:41:49.307007 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:49.307014 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:49.307230 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:49.307408 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:49.308711 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:49.309928 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:49.311161 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:49.311757 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:49.311861 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:49.314519 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:49.314528 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:49.314544 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:49.314579 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:49.317113 (+ 2534us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.30.185.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 18:41:49.317499 (+ 386us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57500: Request ticket server kudu/127.30.185.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":283,"threads_started":1}
W20260811 18:41:49.317895 16749 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57500: Request ticket server kudu/127.30.185.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 18:41:49.317974 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 16185: Not authorized: Client connection negotiation failed: client connection to 127.30.185.130:37627: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.30.185.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 18:41:49.318168 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 16185
I20260811 18:41:49.352087 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 16442
I20260811 18:41:49.365293 16750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:49.353294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41568 (local address 127.30.185.131:33601)
0811 18:41:49.353949 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:49.353957 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:49.353986 (+ 29us) server_negotiation.cc:407] Connection header received
0811 18:41:49.354057 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:49.354064 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:49.354295 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:49.354468 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:49.355940 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:49.357001 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:49.358368 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:49.358931 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:49.359032 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:49.361799 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:49.361807 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:49.361821 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:49.361853 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:49.364615 (+ 2762us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.30.185.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 18:41:49.364928 (+ 313us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41568: Request ticket server kudu/127.30.185.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":359,"threads_started":1}
W20260811 18:41:49.365375 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 16442: Not authorized: Client connection negotiation failed: client connection to 127.30.185.131:33601: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.30.185.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 18:41:49.365430 16750 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41568: Request ticket server kudu/127.30.185.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 18:41:49.365550 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 16442
I20260811 18:41:49.397261 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 15756
I20260811 18:41:49.398700 15873 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:49.519692 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 15756
2026-08-11T18:41:49Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21958 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T18:41:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:41:49Z Disabled control of system clock
I20260811 18:41:49.624053 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:33327
--webserver_interface=127.30.185.190
--webserver_port=0
--builtin_ntp_servers=127.30.185.148:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:33327
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 18:41:49.975279 16762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:49.975586 16762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:49.975668 16762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:49.983099 16762 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:41:49.983224 16762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:49.983269 16762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:49.983320 16762 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:41:49.983367 16762 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:41:49.993106 16762 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:33327
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:33327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:49.994544 16762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:49.996356 16762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:50.008999 16776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:50.009088 16774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:50.009107 16773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:50.009889 16762 server_base.cc:1034] running on GCE node
I20260811 18:41:50.010831 16762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:50.012372 16762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:50.013610 16762 hybrid_clock.cc:648] HybridClock initialized: now 1786473710013550 us; error 68 us; skew 500 ppm
I20260811 18:41:50.013984 16762 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:41:50.016371 16762 webserver.cc:460] Webserver started at http://127.30.185.190:41633/ using document root <none> and password file <none>
I20260811 18:41:50.017277 16762 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:50.017422 16762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:50.017782 16762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:50.020769 16762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "6e60f54ab881434a876c89dfc5738749"
format_stamp: "Formatted at 2026-08-11 18:41:50 on dist-test-slave-z8x9"
server_key: "2ae8ad5e599ae57fd55470eadfb7a19d"
server_key_iv: "4a6cbfb21cac39fd174c6415db37afc4"
server_key_version: "encryptionkey@0"
I20260811 18:41:50.021618 16762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "6e60f54ab881434a876c89dfc5738749"
format_stamp: "Formatted at 2026-08-11 18:41:50 on dist-test-slave-z8x9"
server_key: "2ae8ad5e599ae57fd55470eadfb7a19d"
server_key_iv: "4a6cbfb21cac39fd174c6415db37afc4"
server_key_version: "encryptionkey@0"
I20260811 18:41:50.027099 16762 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 18:41:50.030903 16782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:50.032321 16762 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:41:50.032521 16762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "6e60f54ab881434a876c89dfc5738749"
format_stamp: "Formatted at 2026-08-11 18:41:50 on dist-test-slave-z8x9"
server_key: "2ae8ad5e599ae57fd55470eadfb7a19d"
server_key_iv: "4a6cbfb21cac39fd174c6415db37afc4"
server_key_version: "encryptionkey@0"
I20260811 18:41:50.032775 16762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:50.060448 16762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:50.061506 16762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:50.061842 16762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:50.081384 16762 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:33327
I20260811 18:41:50.081425 16879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:33327 every 8 connection(s)
I20260811 18:41:50.082798 16762 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:41:50.084363 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 16762
I20260811 18:41:50.084587 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:41:50.085498 31462 external_mini_cluster.cc:1428] Setting key 00c2877473b0cf55ff7e5ac0f59d8bb7
I20260811 18:41:50.087854 16881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:50.101580 16884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:50.087711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51780 (local address 127.30.185.190:33327)
0811 18:41:50.088794 (+ 1083us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:50.088812 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:50.088866 (+ 54us) server_negotiation.cc:407] Connection header received
0811 18:41:50.091091 (+ 2225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:50.091119 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:50.091741 (+ 622us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:50.092187 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:50.092724 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:50.094797 (+ 2073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:50.096138 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:50.097200 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:50.097401 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:50.098587 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:50.098598 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:50.098616 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:41:50.099397 (+ 781us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:50.099504 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:41:50.099531 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:50.099648 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:50.099902 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 18:41:50.100254 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":355,"threads_started":1}
I20260811 18:41:50.102854 16881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e60f54ab881434a876c89dfc5738749: Bootstrap starting.
I20260811 18:41:50.106001 16881 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e60f54ab881434a876c89dfc5738749: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:50.107085 16881 log.cc:824] T 00000000000000000000000000000000 P 6e60f54ab881434a876c89dfc5738749: Log is configured to *not* fsync() on all Append() calls
I20260811 18:41:50.109839 16881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e60f54ab881434a876c89dfc5738749: No bootstrap required, opened a new log
I20260811 18:41:50.117045 16881 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e60f54ab881434a876c89dfc5738749 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e60f54ab881434a876c89dfc5738749" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33327 } }
I20260811 18:41:50.117518 16881 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e60f54ab881434a876c89dfc5738749 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:50.117645 16881 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e60f54ab881434a876c89dfc5738749 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e60f54ab881434a876c89dfc5738749, State: Initialized, Role: FOLLOWER
I20260811 18:41:50.118227 16881 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e60f54ab881434a876c89dfc5738749 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e60f54ab881434a876c89dfc5738749" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33327 } }
I20260811 18:41:50.118474 16881 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e60f54ab881434a876c89dfc5738749 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:41:50.118619 16881 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e60f54ab881434a876c89dfc5738749 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:41:50.118808 16881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e60f54ab881434a876c89dfc5738749 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:50.120998 16881 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e60f54ab881434a876c89dfc5738749 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e60f54ab881434a876c89dfc5738749" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33327 } }
I20260811 18:41:50.121539 16881 leader_election.cc:302] T 00000000000000000000000000000000 P 6e60f54ab881434a876c89dfc5738749 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e60f54ab881434a876c89dfc5738749; no voters:
I20260811 18:41:50.122140 16881 leader_election.cc:288] T 00000000000000000000000000000000 P 6e60f54ab881434a876c89dfc5738749 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:41:50.122360 16889 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e60f54ab881434a876c89dfc5738749 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:41:50.123346 16889 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e60f54ab881434a876c89dfc5738749 [term 1 LEADER]: Becoming Leader. State: Replica: 6e60f54ab881434a876c89dfc5738749, State: Running, Role: LEADER
I20260811 18:41:50.123939 16889 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e60f54ab881434a876c89dfc5738749 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e60f54ab881434a876c89dfc5738749" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33327 } }
I20260811 18:41:50.124449 16881 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e60f54ab881434a876c89dfc5738749 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:41:50.127214 16891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e60f54ab881434a876c89dfc5738749 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e60f54ab881434a876c89dfc5738749. Latest consensus state: current_term: 1 leader_uuid: "6e60f54ab881434a876c89dfc5738749" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e60f54ab881434a876c89dfc5738749" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33327 } } }
I20260811 18:41:50.127635 16891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e60f54ab881434a876c89dfc5738749 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:50.128019 16890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e60f54ab881434a876c89dfc5738749 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e60f54ab881434a876c89dfc5738749" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e60f54ab881434a876c89dfc5738749" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33327 } } }
I20260811 18:41:50.128392 16890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e60f54ab881434a876c89dfc5738749 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:50.134749 16904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:41:50.140710 16904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:41:50.149956 16904 catalog_manager.cc:1347] Generated new cluster ID: 1bc27b026b5d492ba2da7fd4f9ec415b
I20260811 18:41:50.150102 16904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:41:50.181491 16904 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:41:50.183493 16904 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:41:50.202883 16904 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e60f54ab881434a876c89dfc5738749: Generated new TSK 0
I20260811 18:41:50.203809 16904 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:41:50.220920 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:33327
--builtin_ntp_servers=127.30.185.148:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 18:41:50.582949 16924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:50.583247 16924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:50.583335 16924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:50.590801 16924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:50.590919 16924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:50.591032 16924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:41:50.600333 16924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:33327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:50.601836 16924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:50.603761 16924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:50.617686 16932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:50.617755 16935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:50.617718 16933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:50.618860 16924 server_base.cc:1034] running on GCE node
I20260811 18:41:50.619445 16924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:50.620332 16924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:50.621537 16924 hybrid_clock.cc:648] HybridClock initialized: now 1786473710621492 us; error 31 us; skew 500 ppm
I20260811 18:41:50.621820 16924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:41:50.624166 16924 webserver.cc:460] Webserver started at http://127.30.185.129:43453/ using document root <none> and password file <none>
I20260811 18:41:50.624867 16924 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:50.624962 16924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:50.625322 16924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:50.628091 16924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "3bc5a9056f234d66920fd21452e543a3"
format_stamp: "Formatted at 2026-08-11 18:41:50 on dist-test-slave-z8x9"
server_key: "e7610bb247c93a9c86382b0e68d2e2f9"
server_key_iv: "fcc642940bbf638be04f89655cf116cd"
server_key_version: "encryptionkey@0"
I20260811 18:41:50.628878 16924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "3bc5a9056f234d66920fd21452e543a3"
format_stamp: "Formatted at 2026-08-11 18:41:50 on dist-test-slave-z8x9"
server_key: "e7610bb247c93a9c86382b0e68d2e2f9"
server_key_iv: "fcc642940bbf638be04f89655cf116cd"
server_key_version: "encryptionkey@0"
I20260811 18:41:50.634213 16924 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 18:41:50.637871 16946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:50.639204 16924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:41:50.639400 16924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "3bc5a9056f234d66920fd21452e543a3"
format_stamp: "Formatted at 2026-08-11 18:41:50 on dist-test-slave-z8x9"
server_key: "e7610bb247c93a9c86382b0e68d2e2f9"
server_key_iv: "fcc642940bbf638be04f89655cf116cd"
server_key_version: "encryptionkey@0"
I20260811 18:41:50.639626 16924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:50.681247 16924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:50.682210 16924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:50.682569 16924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:50.683686 16924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:50.685411 16924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:50.685508 16924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:50.685645 16924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:50.685752 16924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:50.721671 16924 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:45165
I20260811 18:41:50.721773 17173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:45165 every 8 connection(s)
I20260811 18:41:50.723039 16924 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:41:50.728724 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 16924
I20260811 18:41:50.728961 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:41:50.729763 31462 external_mini_cluster.cc:1428] Setting key cd4b21986de310b6ac12012442f8c8d3
I20260811 18:41:50.733319 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:33327
--builtin_ntp_servers=127.30.185.148:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 18:41:50.739037 17178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:50.726303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:55027 (local address 127.30.185.190:33327)
0811 18:41:50.726818 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:50.726826 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:50.728266 (+ 1440us) server_negotiation.cc:407] Connection header received
0811 18:41:50.730261 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:50.730269 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:50.730371 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:50.730514 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:50.732089 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:50.733263 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:50.735206 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:50.735993 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:50.736148 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:50.737913 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:50.737923 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:50.737930 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:41:50.737957 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:50.738017 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:41:50.738037 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:50.738144 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:50.738429 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 18:41:50.738551 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":227,"threads_started":1}
I20260811 18:41:50.740305 17179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:50.726977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:33327 (local address 127.30.185.129:55027)
0811 18:41:50.727989 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:50.728040 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:50.729958 (+ 1918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:50.730811 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:50.730820 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:50.730917 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:50.731863 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:50.731891 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:50.733500 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:50.733510 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:50.735021 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:50.735035 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:50.736269 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:50.736275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:50.736502 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:50.737358 (+ 856us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:41:50.737361 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:50.737406 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:41:50.737499 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:50.738157 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:50.738172 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:50.738183 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 18:41:50.738393 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 18:41:50.738860 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":303,"threads_started":1}
I20260811 18:41:50.742036 17176 heartbeater.cc:347] Connected to a master server at 127.30.185.190:33327
I20260811 18:41:50.742404 17176 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:50.743436 17176 heartbeater.cc:511] Master 127.30.185.190:33327 requested a full tablet report, sending...
I20260811 18:41:50.745992 16815 ts_manager.cc:194] Registered new tserver with Master: 3bc5a9056f234d66920fd21452e543a3 (127.30.185.129:45165)
I20260811 18:41:50.748143 16815 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.185.129:55027
W20260811 18:41:51.078671 17180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:51.078977 17180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:51.079061 17180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:51.086603 17180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:51.086841 17180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:51.087030 17180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:41:51.096273 17180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:33327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:51.097676 17180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:51.099480 17180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:51.112818 17186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:51.112788 17188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:51.113660 17180 server_base.cc:1034] running on GCE node
W20260811 18:41:51.113833 17191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:51.114526 17180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:51.115512 17180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:51.116755 17180 hybrid_clock.cc:648] HybridClock initialized: now 1786473711116701 us; error 48 us; skew 500 ppm
I20260811 18:41:51.117070 17180 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:41:51.119318 17180 webserver.cc:460] Webserver started at http://127.30.185.130:42635/ using document root <none> and password file <none>
I20260811 18:41:51.120055 17180 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:51.120195 17180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:51.120538 17180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:51.123553 17180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "de9df4ea7a104604894d1cf6af49ed8e"
format_stamp: "Formatted at 2026-08-11 18:41:51 on dist-test-slave-z8x9"
server_key: "e9740279fb964c104f0f8eb963da1f03"
server_key_iv: "60b07ec7cce64f550ded71db8b04882c"
server_key_version: "encryptionkey@0"
I20260811 18:41:51.124409 17180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "de9df4ea7a104604894d1cf6af49ed8e"
format_stamp: "Formatted at 2026-08-11 18:41:51 on dist-test-slave-z8x9"
server_key: "e9740279fb964c104f0f8eb963da1f03"
server_key_iv: "60b07ec7cce64f550ded71db8b04882c"
server_key_version: "encryptionkey@0"
I20260811 18:41:51.129889 17180 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 18:41:51.133574 17208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:51.135080 17180 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:41:51.135285 17180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "de9df4ea7a104604894d1cf6af49ed8e"
format_stamp: "Formatted at 2026-08-11 18:41:51 on dist-test-slave-z8x9"
server_key: "e9740279fb964c104f0f8eb963da1f03"
server_key_iv: "60b07ec7cce64f550ded71db8b04882c"
server_key_version: "encryptionkey@0"
I20260811 18:41:51.135521 17180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:51.150391 17180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:51.151284 17180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:51.151599 17180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:51.152585 17180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:51.154171 17180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:51.154309 17180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:51.154440 17180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:51.154534 17180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:51.190675 17180 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:36637
I20260811 18:41:51.190726 17423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:36637 every 8 connection(s)
I20260811 18:41:51.192164 17180 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:41:51.197261 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 17180
I20260811 18:41:51.197502 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:41:51.197944 31462 external_mini_cluster.cc:1428] Setting key c35e2853d1bc663a6525a49349f03529
I20260811 18:41:51.201417 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:33327
--builtin_ntp_servers=127.30.185.148:38999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 18:41:51.207571 17178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:51.194990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:59669 (local address 127.30.185.190:33327)
0811 18:41:51.195209 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:51.195215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:51.196850 (+ 1635us) server_negotiation.cc:407] Connection header received
0811 18:41:51.199234 (+ 2384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:51.199239 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:51.199311 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:51.199391 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:51.200877 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:51.202025 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:51.204129 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:51.204899 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:51.205056 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:51.206580 (+ 1524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:51.206589 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:51.206597 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:41:51.206623 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:51.206668 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:41:51.206687 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:51.206784 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:51.207033 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 18:41:51.207148 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":75}
I20260811 18:41:51.208575 17427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:51.195485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:33327 (local address 127.30.185.130:59669)
0811 18:41:51.196619 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:51.196689 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:51.198894 (+ 2205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:51.199701 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:51.199713 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:51.199845 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:51.200684 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:51.200704 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:51.202211 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:51.202216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:51.203964 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:51.203978 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:51.205162 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:51.205206 (+ 44us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:51.205443 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:51.206224 (+ 781us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:41:51.206227 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:51.206271 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:41:51.206356 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:51.206808 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:51.206817 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:51.206824 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:41:51.206993 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 18:41:51.207341 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":294,"threads_started":1}
I20260811 18:41:51.210194 17425 heartbeater.cc:347] Connected to a master server at 127.30.185.190:33327
I20260811 18:41:51.210536 17425 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:51.211509 17425 heartbeater.cc:511] Master 127.30.185.190:33327 requested a full tablet report, sending...
I20260811 18:41:51.213307 16815 ts_manager.cc:194] Registered new tserver with Master: de9df4ea7a104604894d1cf6af49ed8e (127.30.185.130:36637)
I20260811 18:41:51.214138 16815 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.185.130:59669
W20260811 18:41:51.548161 17429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:51.548466 17429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:51.548614 17429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:51.555852 17429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:51.555956 17429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:51.556129 17429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:41:51.565260 17429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:33327
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:51.566588 17429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:51.568429 17429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:51.582042 17438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:51.582089 17437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:51.582264 17440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:51.586971 17429 server_base.cc:1034] running on GCE node
I20260811 18:41:51.587751 17429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:51.589005 17429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:51.590250 17429 hybrid_clock.cc:648] HybridClock initialized: now 1786473711590202 us; error 59 us; skew 500 ppm
I20260811 18:41:51.590617 17429 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:41:51.593303 17429 webserver.cc:460] Webserver started at http://127.30.185.131:39423/ using document root <none> and password file <none>
I20260811 18:41:51.594089 17429 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:51.594185 17429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:51.594537 17429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:51.598440 17429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "616f15dc30534216897b7c74c8cde88f"
format_stamp: "Formatted at 2026-08-11 18:41:51 on dist-test-slave-z8x9"
server_key: "73f3b3b100c689c9de5c1665ccaeeced"
server_key_iv: "5bd25f13c6ab1ee5bf687a23745b8cf1"
server_key_version: "encryptionkey@0"
I20260811 18:41:51.599352 17429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "616f15dc30534216897b7c74c8cde88f"
format_stamp: "Formatted at 2026-08-11 18:41:51 on dist-test-slave-z8x9"
server_key: "73f3b3b100c689c9de5c1665ccaeeced"
server_key_iv: "5bd25f13c6ab1ee5bf687a23745b8cf1"
server_key_version: "encryptionkey@0"
I20260811 18:41:51.605280 17429 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 18:41:51.609001 17451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:51.610595 17429 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:41:51.610821 17429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "616f15dc30534216897b7c74c8cde88f"
format_stamp: "Formatted at 2026-08-11 18:41:51 on dist-test-slave-z8x9"
server_key: "73f3b3b100c689c9de5c1665ccaeeced"
server_key_iv: "5bd25f13c6ab1ee5bf687a23745b8cf1"
server_key_version: "encryptionkey@0"
I20260811 18:41:51.611084 17429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:51.630337 17429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:51.631325 17429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:51.631675 17429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:51.632850 17429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:51.634591 17429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:51.634725 17429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:51.634866 17429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:51.634960 17429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:51.671279 17429 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:45731
I20260811 18:41:51.671350 17678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:45731 every 8 connection(s)
I20260811 18:41:51.672672 17429 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:41:51.673987 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 17429
I20260811 18:41:51.674198 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:41:51.674829 31462 external_mini_cluster.cc:1428] Setting key 59d9999b2aeca3e3f4763c4fe684c6c7
I20260811 18:41:51.686566 17178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:51.675784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:44253 (local address 127.30.185.190:33327)
0811 18:41:51.676082 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:51.676087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:51.677936 (+ 1849us) server_negotiation.cc:407] Connection header received
0811 18:41:51.679969 (+ 2033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:51.679974 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:51.680062 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:51.680183 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:51.681292 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:51.682066 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:51.683667 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:51.684084 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:51.684222 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:51.685674 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:51.685681 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:51.685686 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:41:51.685714 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:51.685763 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:41:51.685775 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:51.685876 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:51.686113 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 18:41:51.686211 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167}
I20260811 18:41:51.687553 17681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:51.676558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:33327 (local address 127.30.185.131:44253)
0811 18:41:51.677790 (+ 1232us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:51.677828 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:51.679689 (+ 1861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:51.680426 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:51.680434 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:51.680519 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:51.681087 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:51.681103 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:51.682237 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:51.682241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:51.683513 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:51.683526 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:51.684326 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:51.684333 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:51.684565 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:51.685346 (+ 781us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:41:51.685349 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:51.685388 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:41:51.685466 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:51.685879 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:51.685893 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:51.685902 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 18:41:51.686083 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 18:41:51.686488 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":336,"threads_started":1}
I20260811 18:41:51.688942 17679 heartbeater.cc:347] Connected to a master server at 127.30.185.190:33327
I20260811 18:41:51.689358 17679 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:51.690326 17679 heartbeater.cc:511] Master 127.30.185.190:33327 requested a full tablet report, sending...
I20260811 18:41:51.691828 16815 ts_manager.cc:194] Registered new tserver with Master: 616f15dc30534216897b7c74c8cde88f (127.30.185.131:45731)
I20260811 18:41:51.692665 16815 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.185.131:44253
I20260811 18:41:51.703768 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 18:41:51.710336 17178 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:51.708679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51782 (local address 127.30.185.190:33327)
0811 18:41:51.708872 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:51.708877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:51.709408 (+ 531us) server_negotiation.cc:407] Connection header received
0811 18:41:51.709554 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:51.709558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:51.709628 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:51.709718 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:51.710116 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51782: BlockingRecv error: recv got EOF from 127.0.0.1:51782 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 18:41:51.710798 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 16924
I20260811 18:41:51.719066 17690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:51.711884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58924 (local address 127.30.185.129:45165)
0811 18:41:51.712497 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:51.712503 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:51.712530 (+ 27us) server_negotiation.cc:407] Connection header received
0811 18:41:51.712585 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:51.712591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:51.712810 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:51.712934 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:51.713377 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:51.714385 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:51.715806 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:51.716379 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:51.716477 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:51.717612 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:51.717620 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:51.717634 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:41:51.718252 (+ 618us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:51.718300 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:41:51.718319 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:51.718409 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:51.718555 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 18:41:51.718693 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":289,"threads_started":1}
I20260811 18:41:51.720290 17164 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:51.804401 17176 heartbeater.cc:503] Master 127.30.185.190:33327 was elected leader, sending a full tablet report...
I20260811 18:41:51.806430 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 16924
I20260811 18:41:51.833295 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 17180
I20260811 18:41:51.841848 17697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:51.834242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37216 (local address 127.30.185.130:36637)
0811 18:41:51.834849 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:51.834879 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:51.834920 (+ 41us) server_negotiation.cc:407] Connection header received
0811 18:41:51.834998 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:51.835005 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:51.835231 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:51.835376 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:51.835860 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:51.836989 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:51.838395 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:51.839040 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:51.839146 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:51.840298 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:51.840308 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:51.840322 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:41:51.840938 (+ 616us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:51.840988 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:41:51.841008 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:51.841107 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:51.841396 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 18:41:51.841517 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":297,"threads_started":1}
I20260811 18:41:51.843125 17412 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:51.926026 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 17180
I20260811 18:41:51.952715 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 17429
I20260811 18:41:51.961922 17700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:51.953932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35050 (local address 127.30.185.131:45731)
0811 18:41:51.954580 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:51.954587 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:51.954619 (+ 32us) server_negotiation.cc:407] Connection header received
0811 18:41:51.954685 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:51.954692 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:51.954939 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:51.955085 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:51.955527 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:51.956599 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:51.958119 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:51.958851 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:51.959077 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:51.960399 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:51.960408 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:51.960422 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:41:51.961035 (+ 613us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:51.961091 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:41:51.961110 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:51.961324 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:51.961456 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 18:41:51.961586 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":344,"threads_started":1}
I20260811 18:41:51.963254 17671 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:52.047161 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 17429
I20260811 18:41:52.074221 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 16762
I20260811 18:41:52.075477 16874 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:52.180702 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 16762
2026-08-11T18:41:52Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2652 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T18:41:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:41:52Z Disabled control of system clock
I20260811 18:41:52.280215 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:42833
--webserver_interface=127.30.185.190
--webserver_port=0
--builtin_ntp_servers=127.30.185.148:38119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:42833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 18:41:52.623697 17712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:52.624009 17712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:52.624166 17712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:52.631498 17712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:41:52.631613 17712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:52.631651 17712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:52.631765 17712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:41:52.631850 17712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:41:52.641215 17712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:42833
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:42833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:52.642570 17712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:52.644423 17712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:52.656502 17720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:52.656447 17725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:52.656416 17722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:52.657274 17712 server_base.cc:1034] running on GCE node
I20260811 18:41:52.658195 17712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:52.659631 17712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:52.660871 17712 hybrid_clock.cc:648] HybridClock initialized: now 1786473712660813 us; error 61 us; skew 500 ppm
I20260811 18:41:52.661299 17712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:41:52.664429 17712 webserver.cc:460] Webserver started at http://127.30.185.190:42569/ using document root <none> and password file <none>
I20260811 18:41:52.665278 17712 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:52.665427 17712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:52.665845 17712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:52.669970 17712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "6eb522e404b146858c6dd1e5113ee2d4"
format_stamp: "Formatted at 2026-08-11 18:41:52 on dist-test-slave-z8x9"
server_key: "6d88e97cf26c09da144ab228e0b69252"
server_key_iv: "1bf1ee2fcaaa9320e56fa2fc8dedcc2a"
server_key_version: "encryptionkey@0"
I20260811 18:41:52.670882 17712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "6eb522e404b146858c6dd1e5113ee2d4"
format_stamp: "Formatted at 2026-08-11 18:41:52 on dist-test-slave-z8x9"
server_key: "6d88e97cf26c09da144ab228e0b69252"
server_key_iv: "1bf1ee2fcaaa9320e56fa2fc8dedcc2a"
server_key_version: "encryptionkey@0"
I20260811 18:41:52.676192 17712 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 18:41:52.679674 17739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:52.681030 17712 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:41:52.681254 17712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "6eb522e404b146858c6dd1e5113ee2d4"
format_stamp: "Formatted at 2026-08-11 18:41:52 on dist-test-slave-z8x9"
server_key: "6d88e97cf26c09da144ab228e0b69252"
server_key_iv: "1bf1ee2fcaaa9320e56fa2fc8dedcc2a"
server_key_version: "encryptionkey@0"
I20260811 18:41:52.681483 17712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:52.687302 17712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:52.688084 17712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:52.688407 17712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:52.707564 17712 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:42833
I20260811 18:41:52.707600 17825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:42833 every 8 connection(s)
I20260811 18:41:52.708972 17712 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:41:52.709599 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 17712
I20260811 18:41:52.709812 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:41:52.710683 31462 external_mini_cluster.cc:1428] Setting key 47a2c356d84623f03e609802ca9cb878
I20260811 18:41:52.713508 17827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:52.719312 17833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:52.712871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42372 (local address 127.30.185.190:42833)
0811 18:41:52.713963 (+ 1092us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:52.713974 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:52.714018 (+ 44us) server_negotiation.cc:407] Connection header received
0811 18:41:52.715592 (+ 1574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:52.715614 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:52.716042 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:52.716368 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:52.717619 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:52.717625 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:52.717637 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:41:52.717658 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:52.717733 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:41:52.717769 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:52.717939 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:52.718082 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 18:41:52.718375 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":382,"threads_started":1}
I20260811 18:41:52.725425 17827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6eb522e404b146858c6dd1e5113ee2d4: Bootstrap starting.
I20260811 18:41:52.728423 17827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6eb522e404b146858c6dd1e5113ee2d4: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:52.729558 17827 log.cc:824] T 00000000000000000000000000000000 P 6eb522e404b146858c6dd1e5113ee2d4: Log is configured to *not* fsync() on all Append() calls
I20260811 18:41:52.732537 17827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6eb522e404b146858c6dd1e5113ee2d4: No bootstrap required, opened a new log
I20260811 18:41:52.739948 17827 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6eb522e404b146858c6dd1e5113ee2d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb522e404b146858c6dd1e5113ee2d4" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 42833 } }
I20260811 18:41:52.740350 17827 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6eb522e404b146858c6dd1e5113ee2d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:52.740463 17827 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6eb522e404b146858c6dd1e5113ee2d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eb522e404b146858c6dd1e5113ee2d4, State: Initialized, Role: FOLLOWER
I20260811 18:41:52.741027 17827 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6eb522e404b146858c6dd1e5113ee2d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb522e404b146858c6dd1e5113ee2d4" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 42833 } }
I20260811 18:41:52.741312 17827 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6eb522e404b146858c6dd1e5113ee2d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:41:52.741477 17827 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6eb522e404b146858c6dd1e5113ee2d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:41:52.741694 17827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6eb522e404b146858c6dd1e5113ee2d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:52.744004 17827 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6eb522e404b146858c6dd1e5113ee2d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb522e404b146858c6dd1e5113ee2d4" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 42833 } }
I20260811 18:41:52.744576 17827 leader_election.cc:302] T 00000000000000000000000000000000 P 6eb522e404b146858c6dd1e5113ee2d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6eb522e404b146858c6dd1e5113ee2d4; no voters:
I20260811 18:41:52.745326 17827 leader_election.cc:288] T 00000000000000000000000000000000 P 6eb522e404b146858c6dd1e5113ee2d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:41:52.745458 17842 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6eb522e404b146858c6dd1e5113ee2d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:41:52.746204 17842 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6eb522e404b146858c6dd1e5113ee2d4 [term 1 LEADER]: Becoming Leader. State: Replica: 6eb522e404b146858c6dd1e5113ee2d4, State: Running, Role: LEADER
I20260811 18:41:52.746740 17842 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6eb522e404b146858c6dd1e5113ee2d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb522e404b146858c6dd1e5113ee2d4" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 42833 } }
I20260811 18:41:52.747476 17827 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6eb522e404b146858c6dd1e5113ee2d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:41:52.749694 17846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eb522e404b146858c6dd1e5113ee2d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6eb522e404b146858c6dd1e5113ee2d4. Latest consensus state: current_term: 1 leader_uuid: "6eb522e404b146858c6dd1e5113ee2d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb522e404b146858c6dd1e5113ee2d4" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 42833 } } }
I20260811 18:41:52.750160 17846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eb522e404b146858c6dd1e5113ee2d4 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:52.750161 17844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eb522e404b146858c6dd1e5113ee2d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6eb522e404b146858c6dd1e5113ee2d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb522e404b146858c6dd1e5113ee2d4" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 42833 } } }
I20260811 18:41:52.750939 17844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eb522e404b146858c6dd1e5113ee2d4 [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:52.753540 17854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:41:52.757921 17854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:41:52.766552 17854 catalog_manager.cc:1347] Generated new cluster ID: 5ceecde34f854918828528ef7a2124af
I20260811 18:41:52.766691 17854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:41:52.784829 17854 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:41:52.785430 17854 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:41:52.792357 17854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6eb522e404b146858c6dd1e5113ee2d4: Generated new TSK 0
I20260811 18:41:52.793412 17854 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:41:52.811548 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:42833
--builtin_ntp_servers=127.30.185.148:38119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 18:41:53.167685 17876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:53.167995 17876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:53.168089 17876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:53.175699 17876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:53.175817 17876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:53.175930 17876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:41:53.185520 17876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:42833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:53.186913 17876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:53.188673 17876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:53.202723 17888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:53.202768 17885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:53.202845 17884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:53.203626 17876 server_base.cc:1034] running on GCE node
I20260811 18:41:53.204341 17876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:53.205451 17876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:53.206688 17876 hybrid_clock.cc:648] HybridClock initialized: now 1786473713206641 us; error 59 us; skew 500 ppm
I20260811 18:41:53.207034 17876 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:41:53.213454 17876 webserver.cc:460] Webserver started at http://127.30.185.129:45495/ using document root <none> and password file <none>
I20260811 18:41:53.214247 17876 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:53.214395 17876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:53.214751 17876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:53.217881 17876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "866d4e5842ed425aa92bfd740463517e"
format_stamp: "Formatted at 2026-08-11 18:41:53 on dist-test-slave-z8x9"
server_key: "e163d1da4a2b255b4ecbc5185064d588"
server_key_iv: "64beb29be8eea74e9c14ba1ea81bd412"
server_key_version: "encryptionkey@0"
I20260811 18:41:53.218750 17876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "866d4e5842ed425aa92bfd740463517e"
format_stamp: "Formatted at 2026-08-11 18:41:53 on dist-test-slave-z8x9"
server_key: "e163d1da4a2b255b4ecbc5185064d588"
server_key_iv: "64beb29be8eea74e9c14ba1ea81bd412"
server_key_version: "encryptionkey@0"
I20260811 18:41:53.224218 17876 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 18:41:53.227957 17898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:53.229439 17876 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:41:53.229625 17876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "866d4e5842ed425aa92bfd740463517e"
format_stamp: "Formatted at 2026-08-11 18:41:53 on dist-test-slave-z8x9"
server_key: "e163d1da4a2b255b4ecbc5185064d588"
server_key_iv: "64beb29be8eea74e9c14ba1ea81bd412"
server_key_version: "encryptionkey@0"
I20260811 18:41:53.229883 17876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:53.235982 17876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:53.236876 17876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:53.237241 17876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:53.238273 17876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:53.239866 17876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:53.239996 17876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:53.240142 17876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:53.240239 17876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:53.274363 17876 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:36025
I20260811 18:41:53.274456 18089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:36025 every 8 connection(s)
I20260811 18:41:53.275756 17876 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:41:53.284027 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 17876
I20260811 18:41:53.284088 18094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:53.278404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:53241 (local address 127.30.185.190:42833)
0811 18:41:53.278873 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:53.278878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:53.279931 (+ 1053us) server_negotiation.cc:407] Connection header received
0811 18:41:53.281727 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:53.281732 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:53.281824 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:53.281952 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:53.283268 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:53.283274 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:53.283279 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:41:53.283312 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:53.283357 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:41:53.283370 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:53.283514 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:53.283706 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 18:41:53.283785 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":256,"threads_started":1}
I20260811 18:41:53.284309 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:41:53.285058 31462 external_mini_cluster.cc:1428] Setting key cb49fbf060010f7164e1ef327a4effa2
I20260811 18:41:53.285255 18096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:53.278861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:42833 (local address 127.30.185.129:53241)
0811 18:41:53.279763 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:53.279808 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:53.281448 (+ 1640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:53.282195 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:53.282204 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:53.282297 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:53.282958 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:41:53.282960 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:53.282998 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:41:53.283086 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:53.283471 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:53.283483 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:53.283492 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 18:41:53.283792 (+ 300us) client_negotiation.cc:240] Negotiation successful
0811 18:41:53.284067 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":275,"threads_started":1}
I20260811 18:41:53.287189 18090 heartbeater.cc:347] Connected to a master server at 127.30.185.190:42833
I20260811 18:41:53.287528 18090 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:53.288178 18090 heartbeater.cc:511] Master 127.30.185.190:42833 requested a full tablet report, sending...
I20260811 18:41:53.288769 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:42833
--builtin_ntp_servers=127.30.185.148:38119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 18:41:53.290884 17763 ts_manager.cc:194] Registered new tserver with Master: 866d4e5842ed425aa92bfd740463517e (127.30.185.129:36025)
W20260811 18:41:53.640184 18097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:53.640492 18097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:53.640642 18097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:53.647943 18097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:53.648053 18097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:53.648257 18097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:41:53.657922 18097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:42833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:53.659332 18097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:53.661341 18097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:53.674515 18120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:53.674604 18108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:53.674562 18106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:53.675736 18097 server_base.cc:1034] running on GCE node
I20260811 18:41:53.676397 18097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:53.677443 18097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:53.678654 18097 hybrid_clock.cc:648] HybridClock initialized: now 1786473713678609 us; error 35 us; skew 500 ppm
I20260811 18:41:53.678984 18097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:41:53.681448 18097 webserver.cc:460] Webserver started at http://127.30.185.130:33495/ using document root <none> and password file <none>
I20260811 18:41:53.682152 18097 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:53.682245 18097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:53.682628 18097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:53.685681 18097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "36caf96e539f4123b832dd37b22d4945"
format_stamp: "Formatted at 2026-08-11 18:41:53 on dist-test-slave-z8x9"
server_key: "51c1d50a6f996c50bb0641445baedcdb"
server_key_iv: "b3ad9e49dc4062a7cca2bf761680f054"
server_key_version: "encryptionkey@0"
I20260811 18:41:53.686632 18097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "36caf96e539f4123b832dd37b22d4945"
format_stamp: "Formatted at 2026-08-11 18:41:53 on dist-test-slave-z8x9"
server_key: "51c1d50a6f996c50bb0641445baedcdb"
server_key_iv: "b3ad9e49dc4062a7cca2bf761680f054"
server_key_version: "encryptionkey@0"
I20260811 18:41:53.691897 18097 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:41:53.695588 18131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:53.696965 18097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 18:41:53.697216 18097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "36caf96e539f4123b832dd37b22d4945"
format_stamp: "Formatted at 2026-08-11 18:41:53 on dist-test-slave-z8x9"
server_key: "51c1d50a6f996c50bb0641445baedcdb"
server_key_iv: "b3ad9e49dc4062a7cca2bf761680f054"
server_key_version: "encryptionkey@0"
I20260811 18:41:53.697492 18097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:53.703476 18097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:53.704317 18097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:53.704614 18097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:53.705627 18097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:53.707118 18097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:53.707264 18097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:53.707401 18097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:53.707499 18097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:53.742646 18097 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:36519
I20260811 18:41:53.742708 18339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:36519 every 8 connection(s)
I20260811 18:41:53.744036 18097 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:41:53.751140 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 18097
I20260811 18:41:53.751338 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:41:53.752014 31462 external_mini_cluster.cc:1428] Setting key 7bebff2045b3467a912c6b6e7184f6f1
I20260811 18:41:53.752626 18094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:53.746774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:54501 (local address 127.30.185.190:42833)
0811 18:41:53.746967 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:53.746972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:53.748392 (+ 1420us) server_negotiation.cc:407] Connection header received
0811 18:41:53.750172 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:53.750177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:53.750258 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:53.750360 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:53.751873 (+ 1513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:53.751878 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:53.751881 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:41:53.751897 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:53.751931 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:41:53.751942 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:53.752042 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:53.752332 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 18:41:53.752406 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 18:41:53.753783 18344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:53.747272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:42833 (local address 127.30.185.130:54501)
0811 18:41:53.748229 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:53.748266 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:53.749918 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:53.750558 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:53.750565 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:53.750648 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:53.751523 (+ 875us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:41:53.751528 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:53.751581 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:41:53.751696 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:53.752054 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:53.752067 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:53.752076 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 18:41:53.752426 (+ 350us) client_negotiation.cc:240] Negotiation successful
0811 18:41:53.752685 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":322,"threads_started":1}
I20260811 18:41:53.755224 18340 heartbeater.cc:347] Connected to a master server at 127.30.185.190:42833
I20260811 18:41:53.755420 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:42833
--builtin_ntp_servers=127.30.185.148:38119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 18:41:53.755573 18340 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:53.756175 18340 heartbeater.cc:511] Master 127.30.185.190:42833 requested a full tablet report, sending...
I20260811 18:41:53.757721 17763 ts_manager.cc:194] Registered new tserver with Master: 36caf96e539f4123b832dd37b22d4945 (127.30.185.130:36519)
W20260811 18:41:54.102720 18345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:54.103004 18345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:54.103181 18345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:54.110375 18345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:54.110481 18345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:54.110666 18345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:41:54.119665 18345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:42833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:54.120992 18345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:54.122853 18345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:54.136114 18354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:54.136332 18357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:54.136382 18355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:54.137475 18345 server_base.cc:1034] running on GCE node
I20260811 18:41:54.138123 18345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:54.139115 18345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:54.140342 18345 hybrid_clock.cc:648] HybridClock initialized: now 1786473714140262 us; error 72 us; skew 500 ppm
I20260811 18:41:54.140678 18345 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:41:54.143111 18345 webserver.cc:460] Webserver started at http://127.30.185.131:46741/ using document root <none> and password file <none>
I20260811 18:41:54.143862 18345 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:54.143993 18345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:54.144348 18345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:54.147399 18345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "7dbf2b9144704089ae7d22b32623b92f"
format_stamp: "Formatted at 2026-08-11 18:41:54 on dist-test-slave-z8x9"
server_key: "04ec068cf3ca8716221fea78a39df8e1"
server_key_iv: "75282415bef3e394642ec252822c8237"
server_key_version: "encryptionkey@0"
I20260811 18:41:54.148350 18345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "7dbf2b9144704089ae7d22b32623b92f"
format_stamp: "Formatted at 2026-08-11 18:41:54 on dist-test-slave-z8x9"
server_key: "04ec068cf3ca8716221fea78a39df8e1"
server_key_iv: "75282415bef3e394642ec252822c8237"
server_key_version: "encryptionkey@0"
I20260811 18:41:54.153946 18345 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 18:41:54.157691 18371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:54.159039 18345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 18:41:54.159226 18345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "7dbf2b9144704089ae7d22b32623b92f"
format_stamp: "Formatted at 2026-08-11 18:41:54 on dist-test-slave-z8x9"
server_key: "04ec068cf3ca8716221fea78a39df8e1"
server_key_iv: "75282415bef3e394642ec252822c8237"
server_key_version: "encryptionkey@0"
I20260811 18:41:54.159457 18345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:54.165376 18345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:54.166184 18345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:54.166465 18345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:54.167414 18345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:54.168849 18345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:54.168963 18345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:54.169094 18345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:54.169209 18345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:54.204237 18345 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:38747
I20260811 18:41:54.204294 18581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:38747 every 8 connection(s)
I20260811 18:41:54.205703 18345 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:41:54.208298 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 18345
I20260811 18:41:54.208482 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:41:54.209082 31462 external_mini_cluster.cc:1428] Setting key 2ec62ca6d9e0ad3c0835c05289b7d2cb
I20260811 18:41:54.216316 18094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:54.210389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:43087 (local address 127.30.185.190:42833)
0811 18:41:54.210600 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:54.210607 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:54.212089 (+ 1482us) server_negotiation.cc:407] Connection header received
0811 18:41:54.214034 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:54.214040 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:54.214111 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:54.214229 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:54.215541 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:54.215547 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:54.215552 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:41:54.215572 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:54.215611 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:41:54.215624 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:54.215756 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:54.215947 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 18:41:54.216019 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 18:41:54.217338 18585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:54.210978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:42833 (local address 127.30.185.131:43087)
0811 18:41:54.211934 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:54.211978 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:54.213778 (+ 1800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:54.214466 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:54.214475 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:54.214564 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:54.215210 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:41:54.215212 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:54.215281 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:41:54.215367 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:54.215721 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:54.215732 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:54.215739 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:41:54.216029 (+ 290us) client_negotiation.cc:240] Negotiation successful
0811 18:41:54.216287 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":310,"threads_started":1}
I20260811 18:41:54.219027 18583 heartbeater.cc:347] Connected to a master server at 127.30.185.190:42833
I20260811 18:41:54.219354 18583 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:54.219946 18583 heartbeater.cc:511] Master 127.30.185.190:42833 requested a full tablet report, sending...
I20260811 18:41:54.221418 17763 ts_manager.cc:194] Registered new tserver with Master: 7dbf2b9144704089ae7d22b32623b92f (127.30.185.131:38747)
I20260811 18:41:54.224819 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 18:41:54.231196 18094 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:41:54.229502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42380 (local address 127.30.185.190:42833)
0811 18:41:54.229678 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:54.229683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:54.230213 (+ 530us) server_negotiation.cc:407] Connection header received
0811 18:41:54.230419 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:54.230424 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:54.230487 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:54.230590 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:54.230978 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42380: BlockingRecv error: recv got EOF from 127.0.0.1:42380 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:41:54.232057 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 17876
I20260811 18:41:54.235875 18594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:54.233064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38930 (local address 127.30.185.129:36025)
0811 18:41:54.233667 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:54.233679 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:54.233704 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:41:54.233811 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:54.233819 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:54.234036 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:54.234156 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:54.235142 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:54.235148 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:54.235159 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:41:54.235178 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:54.235220 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:41:54.235234 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:54.235341 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:54.235426 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 18:41:54.235533 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":336,"threads_started":1}
I20260811 18:41:54.237087 18085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:54.317320 18090 heartbeater.cc:503] Master 127.30.185.190:42833 was elected leader, sending a full tablet report...
I20260811 18:41:54.319180 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 17876
I20260811 18:41:54.346276 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 18097
I20260811 18:41:54.349994 18603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:54.347250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34280 (local address 127.30.185.130:36519)
0811 18:41:54.347754 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:54.347761 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:54.347789 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:41:54.347867 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:54.347873 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:54.348137 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:54.348310 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:54.349296 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:54.349305 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:54.349316 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:41:54.349327 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:54.349373 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:41:54.349391 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:54.349495 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:54.349575 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 18:41:54.349690 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":234,"threads_started":1}
I20260811 18:41:54.351262 18330 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:54.435128 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 18097
I20260811 18:41:54.464077 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 18345
I20260811 18:41:54.469520 18608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:54.465611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50404 (local address 127.30.185.131:38747)
0811 18:41:54.466483 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:54.466491 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:54.466522 (+ 31us) server_negotiation.cc:407] Connection header received
0811 18:41:54.466589 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:54.466596 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:54.466870 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:54.467026 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:54.468374 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:54.468383 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:54.468399 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:41:54.468445 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:54.468499 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:41:54.468521 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:54.468698 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:54.468796 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 18:41:54.468916 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":492,"threads_started":1}
I20260811 18:41:54.471323 18576 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:54.591243 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 18345
I20260811 18:41:54.619946 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 17712
I20260811 18:41:54.621374 17815 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:54.730506 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 17712
2026-08-11T18:41:54Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2549 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:41:54 dist-test-slave-z8x9 krb5kdc[18622](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:41:54 dist-test-slave-z8x9 krb5kdc[18622](info): set up 2 sockets
Aug 11 18:41:54 dist-test-slave-z8x9 krb5kdc[18622](info): commencing operation
krb5kdc: starting...
W20260811 18:41:56.188629 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.407s user 0.002s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:41:56 dist-test-slave-z8x9 krb5kdc[18622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:41:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:41:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:41:56.509692 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:41193
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:37369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:41193
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:41:56.858781 18647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:56.859102 18647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:56.859297 18647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:56.866779 18647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:41:56.866912 18647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:56.867031 18647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:56.867131 18647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:41:56.867221 18647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:41:56.877452 18647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:37369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:41193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:41193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:56.878906 18647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:56.880307 18647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:56.893150 18658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:56.893281 18662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:56.893008 18660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:56.894559 18647 server_base.cc:1034] running on GCE node
I20260811 18:41:56.895433 18647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:56.897042 18647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:56.898492 18647 hybrid_clock.cc:648] HybridClock initialized: now 1786473716898283 us; error 92 us; skew 500 ppm
I20260811 18:41:56.898937 18647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:56 dist-test-slave-z8x9 krb5kdc[18622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473716, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:56.903079 18647 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:41:56.904875 18647 webserver.cc:460] Webserver started at http://127.30.185.190:43467/ using document root <none> and password file <none>
I20260811 18:41:56.905696 18647 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:56.905810 18647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:56.906092 18647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:56.909283 18647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "b84e8fc033d541e8b492c74ebcf1fe9e"
format_stamp: "Formatted at 2026-08-11 18:41:56 on dist-test-slave-z8x9"
server_key: "42b46af64978281f60d9c212f115ef30"
server_key_iv: "f29deda0d45774973bc529ab06382c09"
server_key_version: "encryptionkey@0"
I20260811 18:41:56.910096 18647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "b84e8fc033d541e8b492c74ebcf1fe9e"
format_stamp: "Formatted at 2026-08-11 18:41:56 on dist-test-slave-z8x9"
server_key: "42b46af64978281f60d9c212f115ef30"
server_key_iv: "f29deda0d45774973bc529ab06382c09"
server_key_version: "encryptionkey@0"
I20260811 18:41:56.915796 18647 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 18:41:56.919741 18675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:56.921411 18647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:41:56.921641 18647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "b84e8fc033d541e8b492c74ebcf1fe9e"
format_stamp: "Formatted at 2026-08-11 18:41:56 on dist-test-slave-z8x9"
server_key: "42b46af64978281f60d9c212f115ef30"
server_key_iv: "f29deda0d45774973bc529ab06382c09"
server_key_version: "encryptionkey@0"
I20260811 18:41:56.921854 18647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:56.939019 18647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:56.944037 18647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:56.944362 18647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:56.965728 18647 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:41193
I20260811 18:41:56.965752 18768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:41193 every 8 connection(s)
I20260811 18:41:56.967422 18647 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:41:56.971041 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 18647
I20260811 18:41:56.971284 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:41:56.971928 18769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:56.972200 31462 external_mini_cluster.cc:1428] Setting key 689e40dc635202354af3e838db3fc51a
I20260811 18:41:56.987725 18769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e: Bootstrap starting.
Aug 11 18:41:56 dist-test-slave-z8x9 krb5kdc[18622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:41:56.991078 18769 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:56.992194 18769 log.cc:824] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e: Log is configured to *not* fsync() on all Append() calls
I20260811 18:41:56.995175 18769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e: No bootstrap required, opened a new log
I20260811 18:41:56.996351 18776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:56.974491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40466 (local address 127.30.185.190:41193)
0811 18:41:56.975410 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:56.975422 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:56.975471 (+ 49us) server_negotiation.cc:407] Connection header received
0811 18:41:56.977014 (+ 1543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:56.977037 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:56.977526 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:56.977869 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:56.979848 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:56.981281 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:56.983111 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:56.983823 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:56.983956 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:56.988890 (+ 4934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:56.988899 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:56.988913 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:56.988956 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:56.992232 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:56.992967 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:56.992978 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:56.992984 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:56.993156 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:56.993620 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:56.993626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:56.993629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:56.994146 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:56.994485 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:56.994912 (+ 427us) server_negotiation.cc:299] Negotiation successful
0811 18:41:56.995372 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":433,"threads_started":1}
I20260811 18:41:57.003129 18769 raft_consensus.cc:348] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84e8fc033d541e8b492c74ebcf1fe9e" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41193 } }
I20260811 18:41:57.003536 18769 raft_consensus.cc:374] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:57.003639 18769 raft_consensus.cc:729] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b84e8fc033d541e8b492c74ebcf1fe9e, State: Initialized, Role: FOLLOWER
I20260811 18:41:57.004321 18769 consensus_queue.cc:260] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84e8fc033d541e8b492c74ebcf1fe9e" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41193 } }
I20260811 18:41:57.004534 18769 raft_consensus.cc:388] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:41:57.004699 18769 raft_consensus.cc:482] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:41:57.004890 18769 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:57.007357 18769 raft_consensus.cc:504] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84e8fc033d541e8b492c74ebcf1fe9e" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41193 } }
I20260811 18:41:57.007923 18769 leader_election.cc:302] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b84e8fc033d541e8b492c74ebcf1fe9e; no voters:
I20260811 18:41:57.008548 18769 leader_election.cc:288] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:41:57.008703 18779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:41:57.009647 18779 raft_consensus.cc:686] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e [term 1 LEADER]: Becoming Leader. State: Replica: b84e8fc033d541e8b492c74ebcf1fe9e, State: Running, Role: LEADER
I20260811 18:41:57.010263 18779 consensus_queue.cc:237] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84e8fc033d541e8b492c74ebcf1fe9e" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41193 } }
I20260811 18:41:57.011020 18769 sys_catalog.cc:564] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:41:57.013329 18781 sys_catalog.cc:455] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b84e8fc033d541e8b492c74ebcf1fe9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84e8fc033d541e8b492c74ebcf1fe9e" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41193 } } }
I20260811 18:41:57.013789 18781 sys_catalog.cc:458] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:57.013334 18782 sys_catalog.cc:455] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b84e8fc033d541e8b492c74ebcf1fe9e. Latest consensus state: current_term: 1 leader_uuid: "b84e8fc033d541e8b492c74ebcf1fe9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b84e8fc033d541e8b492c74ebcf1fe9e" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41193 } } }
I20260811 18:41:57.014704 18782 sys_catalog.cc:458] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e [sys.catalog]: This master's current role is: LEADER
I20260811 18:41:57.015514 18789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:41:57.020941 18789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:41:57.030859 18789 catalog_manager.cc:1347] Generated new cluster ID: 01f1d1449c6c41888add7a913c4cf5b8
I20260811 18:41:57.031010 18789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:41:57.045413 18789 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:41:57.047237 18789 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:41:57.054419 18789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e: Generated new TSK 0
I20260811 18:41:57.055312 18789 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.185.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
I20260811 18:41:57.131626 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41193
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:37369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:41:57.483124 18823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:57.483429 18823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:57.483593 18823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:57.491207 18823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:57.491336 18823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:57.491541 18823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:41:57.501797 18823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:37369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:57.503176 18823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:57.504464 18823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:57.518123 18831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:57.518241 18832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:57.518215 18836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:57.519150 18823 server_base.cc:1034] running on GCE node
I20260811 18:41:57.519742 18823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:57.520690 18823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:57.521919 18823 hybrid_clock.cc:648] HybridClock initialized: now 1786473717521869 us; error 36 us; skew 500 ppm
I20260811 18:41:57.522271 18823 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:57 dist-test-slave-z8x9 krb5kdc[18622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473717, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:57.525564 18823 init.cc:377] Logged in from keytab as kudu/127.30.185.129@KRBTEST.COM (short username kudu)
I20260811 18:41:57.527276 18823 webserver.cc:460] Webserver started at http://127.30.185.129:42457/ using document root <none> and password file <none>
I20260811 18:41:57.527992 18823 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:57.528146 18823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:57.528465 18823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:57.531491 18823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "5427cf526a8b40ef82b77db5031ee755"
format_stamp: "Formatted at 2026-08-11 18:41:57 on dist-test-slave-z8x9"
server_key: "aff525f8c66f5f083bff6084338edf31"
server_key_iv: "0863965553744f5dad77d0c34816907f"
server_key_version: "encryptionkey@0"
I20260811 18:41:57.532290 18823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "5427cf526a8b40ef82b77db5031ee755"
format_stamp: "Formatted at 2026-08-11 18:41:57 on dist-test-slave-z8x9"
server_key: "aff525f8c66f5f083bff6084338edf31"
server_key_iv: "0863965553744f5dad77d0c34816907f"
server_key_version: "encryptionkey@0"
I20260811 18:41:57.537600 18823 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 18:41:57.541049 18856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:57.542508 18823 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:41:57.542702 18823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "5427cf526a8b40ef82b77db5031ee755"
format_stamp: "Formatted at 2026-08-11 18:41:57 on dist-test-slave-z8x9"
server_key: "aff525f8c66f5f083bff6084338edf31"
server_key_iv: "0863965553744f5dad77d0c34816907f"
server_key_version: "encryptionkey@0"
I20260811 18:41:57.542927 18823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:57.570842 18823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:57.575269 18823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:57.575604 18823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:57.576604 18823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:57.578054 18823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:57.578162 18823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:57.578320 18823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:57.578394 18823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:57.613435 18823 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:38861
I20260811 18:41:57.613489 19054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:38861 every 8 connection(s)
I20260811 18:41:57.614894 18823 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:41:57.624640 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 18823
I20260811 18:41:57.624897 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:41:57.625689 31462 external_mini_cluster.cc:1428] Setting key 85df0fd2ec45752211d54aae19a4f51b
Aug 11 18:41:57 dist-test-slave-z8x9 krb5kdc[18622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473717, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:41:57.635946 19061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:57.617614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:51607 (local address 127.30.185.190:41193)
0811 18:41:57.618102 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:57.618113 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:57.619002 (+ 889us) server_negotiation.cc:407] Connection header received
0811 18:41:57.620832 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:57.620838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:57.620934 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:57.621058 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:57.622583 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:57.623305 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:57.624930 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:57.625474 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:57.625619 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:57.629833 (+ 4214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:57.629840 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:57.629844 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:57.629878 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:57.632607 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:57.633479 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:57.633486 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:57.633489 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:57.633619 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:57.634091 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:57.634097 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:57.634100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:57.634386 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:57.634544 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:57.635314 (+ 770us) server_negotiation.cc:299] Negotiation successful
0811 18:41:57.635557 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":287,"threads_started":1}
I20260811 18:41:57.637029 19065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:57.618124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41193 (local address 127.30.185.129:51607)
0811 18:41:57.618842 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:57.618884 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:57.620539 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:57.621325 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:57.621333 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:57.621847 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:57.622409 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:57.622425 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:57.623487 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:57.623491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:57.624720 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:57.624733 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:57.625739 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:57.625750 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:57.626049 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:57.626925 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:57.626955 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:57.629595 (+ 2640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:57.632773 (+ 3178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:57.632790 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:57.632808 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:57.633357 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:57.633737 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:57.633742 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:57.633745 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:57.633968 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:57.634519 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:57.634527 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:57.634905 (+ 378us) client_negotiation.cc:769] Sending connection context
0811 18:41:57.635188 (+ 283us) client_negotiation.cc:240] Negotiation successful
0811 18:41:57.635762 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":285,"threads_started":1}
I20260811 18:41:57.638741 19059 heartbeater.cc:347] Connected to a master server at 127.30.185.190:41193
I20260811 18:41:57.639063 19059 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:57.639802 19059 heartbeater.cc:511] Master 127.30.185.190:41193 requested a full tablet report, sending...
I20260811 18:41:57.641932 18708 ts_manager.cc:194] Registered new tserver with Master: 5427cf526a8b40ef82b77db5031ee755 (127.30.185.129:38861)
I20260811 18:41:57.643393 18708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:51607
WARNING: no policy specified for kudu/127.30.185.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
I20260811 18:41:57.696089 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41193
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:37369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:41:58.059788 19069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:58.060086 19069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:58.060261 19069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:58.067588 19069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:58.067709 19069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:58.067900 19069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:41:58.077847 19069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:37369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:58.079257 19069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:58.080639 19069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:41:58.094753 19076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:58.094702 19080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:58.094789 19077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:58.096114 19069 server_base.cc:1034] running on GCE node
I20260811 18:41:58.096906 19069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:58.098133 19069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:58.099398 19069 hybrid_clock.cc:648] HybridClock initialized: now 1786473718099327 us; error 59 us; skew 500 ppm
I20260811 18:41:58.099797 19069 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:58 dist-test-slave-z8x9 krb5kdc[18622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473718, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:58.103389 19069 init.cc:377] Logged in from keytab as kudu/127.30.185.130@KRBTEST.COM (short username kudu)
I20260811 18:41:58.105032 19069 webserver.cc:460] Webserver started at http://127.30.185.130:40415/ using document root <none> and password file <none>
I20260811 18:41:58.105855 19069 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:58.105999 19069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:58.106352 19069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:58.109603 19069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "c5494fbb9d7d4e43b699f934eb6ba6c4"
format_stamp: "Formatted at 2026-08-11 18:41:58 on dist-test-slave-z8x9"
server_key: "62706726bcad21e4e566dc00dc075b04"
server_key_iv: "94acf9dc841856d602d44ca556556bfd"
server_key_version: "encryptionkey@0"
I20260811 18:41:58.110442 19069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "c5494fbb9d7d4e43b699f934eb6ba6c4"
format_stamp: "Formatted at 2026-08-11 18:41:58 on dist-test-slave-z8x9"
server_key: "62706726bcad21e4e566dc00dc075b04"
server_key_iv: "94acf9dc841856d602d44ca556556bfd"
server_key_version: "encryptionkey@0"
I20260811 18:41:58.116225 19069 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:41:58.120097 19092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:58.121618 19069 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:41:58.121775 19069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "c5494fbb9d7d4e43b699f934eb6ba6c4"
format_stamp: "Formatted at 2026-08-11 18:41:58 on dist-test-slave-z8x9"
server_key: "62706726bcad21e4e566dc00dc075b04"
server_key_iv: "94acf9dc841856d602d44ca556556bfd"
server_key_version: "encryptionkey@0"
I20260811 18:41:58.122025 19069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:58.139503 19069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:58.143764 19069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:58.144097 19069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:58.145107 19069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:58.146687 19069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:58.146764 19069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:58.146937 19069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:58.146991 19069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:58.181933 19069 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:39633
I20260811 18:41:58.181959 19283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:39633 every 8 connection(s)
I20260811 18:41:58.183290 19069 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:41:58.190153 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 19069
I20260811 18:41:58.190358 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:41:58.190977 31462 external_mini_cluster.cc:1428] Setting key 485a4d0c96870bcecf4cf62af62d712e
Aug 11 18:41:58 dist-test-slave-z8x9 krb5kdc[18622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473718, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:41:58.206604 19294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:58.185992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:52543 (local address 127.30.185.190:41193)
0811 18:41:58.186480 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:58.186487 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:58.187661 (+ 1174us) server_negotiation.cc:407] Connection header received
0811 18:41:58.189500 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:58.189505 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:58.189593 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:58.189721 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:58.191900 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.192684 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.195030 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.195467 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.195570 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:58.200361 (+ 4791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:58.200368 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:58.200371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:58.200409 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:58.203299 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:58.204114 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:58.204119 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:58.204121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:58.204268 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:58.204737 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:58.204741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:58.204744 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:58.205011 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:58.205207 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:58.205958 (+ 751us) server_negotiation.cc:299] Negotiation successful
0811 18:41:58.206209 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":270,"threads_started":1}
I20260811 18:41:58.207656 19297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:58.186722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41193 (local address 127.30.185.130:52543)
0811 18:41:58.187492 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:58.187532 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:58.189158 (+ 1626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:58.189998 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:58.190010 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:58.190833 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:58.191669 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:58.191692 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.192873 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.192878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:58.194858 (+ 1980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:58.194877 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.195711 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.195722 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:58.196073 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:58.196925 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:58.196961 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:58.200078 (+ 3117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:58.203470 (+ 3392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:58.203479 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:58.203498 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:58.203966 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:58.204388 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:58.204393 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:58.204396 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:58.204604 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:58.205146 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:58.205154 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:58.205559 (+ 405us) client_negotiation.cc:769] Sending connection context
0811 18:41:58.205858 (+ 299us) client_negotiation.cc:240] Negotiation successful
0811 18:41:58.206424 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":304,"threads_started":1}
I20260811 18:41:58.209240 19290 heartbeater.cc:347] Connected to a master server at 127.30.185.190:41193
I20260811 18:41:58.209589 19290 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:58.210594 19290 heartbeater.cc:511] Master 127.30.185.190:41193 requested a full tablet report, sending...
I20260811 18:41:58.212395 18708 ts_manager.cc:194] Registered new tserver with Master: c5494fbb9d7d4e43b699f934eb6ba6c4 (127.30.185.130:39633)
I20260811 18:41:58.213689 18708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:52543
WARNING: no policy specified for kudu/127.30.185.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
I20260811 18:41:58.262850 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41193
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:37369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:41:58.615681 19301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:41:58.616029 19301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:41:58.616212 19301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:41:58.624652 19301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:41:58.624815 19301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:41:58.625017 19301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:41:58.635331 19301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:37369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:41:58.636736 19301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:41:58.638188 19301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:41:58.646373 19059 heartbeater.cc:503] Master 127.30.185.190:41193 was elected leader, sending a full tablet report...
W20260811 18:41:58.652107 19314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:58.652263 19319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:41:58.652235 19322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:41:58.653198 19301 server_base.cc:1034] running on GCE node
I20260811 18:41:58.653828 19301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:41:58.654729 19301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:41:58.655937 19301 hybrid_clock.cc:648] HybridClock initialized: now 1786473718655884 us; error 42 us; skew 500 ppm
I20260811 18:41:58.656270 19301 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:41:58 dist-test-slave-z8x9 krb5kdc[18622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473718, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:41:58.664186 19301 init.cc:377] Logged in from keytab as kudu/127.30.185.131@KRBTEST.COM (short username kudu)
I20260811 18:41:58.666236 19301 webserver.cc:460] Webserver started at http://127.30.185.131:34163/ using document root <none> and password file <none>
I20260811 18:41:58.667168 19301 fs_manager.cc:359] Metadata directory not provided
I20260811 18:41:58.667311 19301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:41:58.667819 19301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:41:58.671645 19301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "0703326c1ec04ec794bb9e3a03386920"
format_stamp: "Formatted at 2026-08-11 18:41:58 on dist-test-slave-z8x9"
server_key: "cbcd99945fe35937c6e46f304e10bfed"
server_key_iv: "bcde1b50f694c686178a71ebe7a1f7d3"
server_key_version: "encryptionkey@0"
I20260811 18:41:58.672474 19301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "0703326c1ec04ec794bb9e3a03386920"
format_stamp: "Formatted at 2026-08-11 18:41:58 on dist-test-slave-z8x9"
server_key: "cbcd99945fe35937c6e46f304e10bfed"
server_key_iv: "bcde1b50f694c686178a71ebe7a1f7d3"
server_key_version: "encryptionkey@0"
I20260811 18:41:58.677842 19301 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 18:41:58.681828 19336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:58.683144 19301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:41:58.683329 19301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "0703326c1ec04ec794bb9e3a03386920"
format_stamp: "Formatted at 2026-08-11 18:41:58 on dist-test-slave-z8x9"
server_key: "cbcd99945fe35937c6e46f304e10bfed"
server_key_iv: "bcde1b50f694c686178a71ebe7a1f7d3"
server_key_version: "encryptionkey@0"
I20260811 18:41:58.683528 19301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:41:58.695755 19301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:41:58.700397 19301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:41:58.700732 19301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:41:58.701776 19301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:41:58.703109 19301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:41:58.703220 19301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:58.703374 19301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:41:58.703449 19301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:41:58.738056 19301 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:42899
I20260811 18:41:58.738087 19545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:42899 every 8 connection(s)
I20260811 18:41:58.739432 19301 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:41:58.744269 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 19301
I20260811 18:41:58.744477 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:41:58.745066 31462 external_mini_cluster.cc:1428] Setting key e1e7b3be75c9731decce451a643a95c7
Aug 11 18:41:58 dist-test-slave-z8x9 krb5kdc[18622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473718, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:41:58.760936 19553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:58.742033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:35673 (local address 127.30.185.190:41193)
0811 18:41:58.742571 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:58.742577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:58.743521 (+ 944us) server_negotiation.cc:407] Connection header received
0811 18:41:58.746112 (+ 2591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:58.746116 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:58.746272 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:58.746402 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:58.748503 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.749406 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.751034 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.751477 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.751585 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:58.755096 (+ 3511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:58.755101 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:58.755104 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:58.755137 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:58.757687 (+ 2550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:58.758452 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:58.758457 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:58.758459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:58.758591 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:58.759050 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:58.759057 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:58.759061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:58.759361 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:58.759549 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:58.760269 (+ 720us) server_negotiation.cc:299] Negotiation successful
0811 18:41:58.760541 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":243,"threads_started":1}
I20260811 18:41:58.762012 19554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:58.742591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41193 (local address 127.30.185.131:35673)
0811 18:41:58.743345 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:58.743384 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:58.745770 (+ 2386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:58.746750 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:58.746764 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:58.747593 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:41:58.748295 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:58.748348 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.749568 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.749572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:58.750807 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:58.750825 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.751694 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.751699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:58.751901 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:58.752590 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:41:58.752613 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:41:58.754895 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:41:58.757858 (+ 2963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:58.757865 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:58.757878 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:58.758338 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:58.758694 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:41:58.758697 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:41:58.758700 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:41:58.758909 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:41:58.759508 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:41:58.759516 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:41:58.759889 (+ 373us) client_negotiation.cc:769] Sending connection context
0811 18:41:58.760171 (+ 282us) client_negotiation.cc:240] Negotiation successful
0811 18:41:58.760732 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":288,"threads_started":1}
I20260811 18:41:58.763474 19548 heartbeater.cc:347] Connected to a master server at 127.30.185.190:41193
I20260811 18:41:58.763833 19548 heartbeater.cc:464] Registering TS with master...
I20260811 18:41:58.764599 19548 heartbeater.cc:511] Master 127.30.185.190:41193 requested a full tablet report, sending...
I20260811 18:41:58.766155 18708 ts_manager.cc:194] Registered new tserver with Master: 0703326c1ec04ec794bb9e3a03386920 (127.30.185.131:42899)
I20260811 18:41:58.766939 18708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:35673
I20260811 18:41:58.773512 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:41:58.791169 19553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:58.778259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40482 (local address 127.30.185.190:41193)
0811 18:41:58.778457 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:58.778462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:58.778957 (+ 495us) server_negotiation.cc:407] Connection header received
0811 18:41:58.779104 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:58.779108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:58.779174 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:58.779263 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:58.780755 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.781649 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.782893 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.783279 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.783380 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:58.785845 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:58.785850 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:58.785854 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:58.785889 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:58.788639 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:58.789355 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:58.789359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:58.789361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:58.789489 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:58.789817 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:58.789820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:58.789822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:58.790146 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:58.790350 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:58.790644 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 18:41:58.790818 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 18:41:58.795477 18708 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40482:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:41:58.802940 18708 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:41:58.842523 19577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:58.815295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:42899 (local address 127.0.0.1:51292)
0811 18:41:58.825547 (+ 10252us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:58.825608 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:58.826699 (+ 1091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:58.828100 (+ 1401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:58.828109 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:58.828139 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:58.828605 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:58.828618 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.832916 (+ 4298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.832932 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:58.836238 (+ 3306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:58.836263 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.840980 (+ 4717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.840990 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:58.841307 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:58.841548 (+ 241us) client_negotiation.cc:769] Sending connection context
0811 18:41:58.841707 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 18:41:58.841963 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9891,"mutex_wait_us":121,"thread_start_us":13218,"threads_started":1}
I20260811 18:41:58.842908 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:58.815687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51292 (local address 127.30.185.131:42899)
0811 18:41:58.827488 (+ 11801us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:58.827495 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:58.827528 (+ 33us) server_negotiation.cc:407] Connection header received
0811 18:41:58.827603 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:58.827664 (+ 61us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:58.827889 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:58.828097 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:58.828866 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.831049 (+ 2183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.836527 (+ 5478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.840286 (+ 3759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.840536 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:58.840792 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:58.842353 (+ 1561us) server_negotiation.cc:299] Negotiation successful
0811 18:41:58.842533 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11611,"thread_start_us":11400,"threads_started":1}
I20260811 18:41:58.847162 19575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:58.815888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:38861 (local address 127.0.0.1:34194)
0811 18:41:58.828834 (+ 12946us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:58.828852 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:58.828965 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:58.836939 (+ 7974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:58.836947 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:58.836992 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:58.837662 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:58.837679 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.840615 (+ 2936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.840624 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:58.843351 (+ 2727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:58.843370 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.846255 (+ 2885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.846262 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:58.846443 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:58.846518 (+ 75us) client_negotiation.cc:769] Sending connection context
0811 18:41:58.846592 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 18:41:58.846749 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12796,"mutex_wait_us":35,"thread_start_us":9894,"threads_started":1}
I20260811 18:41:58.847457 19418 tablet_service.cc:1467] Processing CreateTablet for tablet 0e9823a128bf4401bc0a4423e2a938aa (DEFAULT_TABLE table=test-table [id=cc97b894de9a4164bd8fa115c1af3c36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:41:58.848318 19578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:58.816594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34194 (local address 127.30.185.129:38861)
0811 18:41:58.835787 (+ 19193us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:58.835800 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:58.835844 (+ 44us) server_negotiation.cc:407] Connection header received
0811 18:41:58.835976 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:58.835989 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:58.836364 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:58.837960 (+ 1596us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:58.838293 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.840402 (+ 2109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.843537 (+ 3135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.845141 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.847580 (+ 2439us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:58.847783 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:58.847856 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 18:41:58.848008 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18925,"thread_start_us":19969,"threads_started":1}
I20260811 18:41:58.849160 19418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e9823a128bf4401bc0a4423e2a938aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:58.850960 19579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:58.829372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:39633 (local address 127.0.0.1:54526)
0811 18:41:58.842286 (+ 12914us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:58.842307 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:58.842558 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:58.843548 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:58.843552 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:58.843581 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:58.843912 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:58.843923 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.845438 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.845443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:58.847739 (+ 2296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:58.847755 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.849947 (+ 2192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.849954 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:58.850121 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:58.850155 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 18:41:58.850268 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 18:41:58.850401 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12778,"mutex_wait_us":4,"thread_start_us":12577,"threads_started":1}
I20260811 18:41:58.851899 19581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:58.829405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54526 (local address 127.30.185.130:39633)
0811 18:41:58.841583 (+ 12178us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:58.841592 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:58.842427 (+ 835us) server_negotiation.cc:407] Connection header received
0811 18:41:58.842779 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:58.842787 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:58.843060 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:58.843268 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:58.844108 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.845244 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.847928 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.849460 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.849641 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:58.849798 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:58.851366 (+ 1568us) server_negotiation.cc:299] Negotiation successful
0811 18:41:58.851545 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11962,"thread_start_us":12175,"threads_started":1}
I20260811 18:41:58.851539 18943 tablet_service.cc:1467] Processing CreateTablet for tablet 0e9823a128bf4401bc0a4423e2a938aa (DEFAULT_TABLE table=test-table [id=cc97b894de9a4164bd8fa115c1af3c36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:41:58.853049 18943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e9823a128bf4401bc0a4423e2a938aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:58.855916 19177 tablet_service.cc:1467] Processing CreateTablet for tablet 0e9823a128bf4401bc0a4423e2a938aa (DEFAULT_TABLE table=test-table [id=cc97b894de9a4164bd8fa115c1af3c36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:41:58.858958 19177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e9823a128bf4401bc0a4423e2a938aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:41:58.864826 19582 tablet_bootstrap.cc:492] T 0e9823a128bf4401bc0a4423e2a938aa P 5427cf526a8b40ef82b77db5031ee755: Bootstrap starting.
I20260811 18:41:58.867245 19585 tablet_bootstrap.cc:492] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920: Bootstrap starting.
I20260811 18:41:58.869220 19582 tablet_bootstrap.cc:654] T 0e9823a128bf4401bc0a4423e2a938aa P 5427cf526a8b40ef82b77db5031ee755: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:58.870875 19585 tablet_bootstrap.cc:654] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:58.870826 19582 log.cc:824] T 0e9823a128bf4401bc0a4423e2a938aa P 5427cf526a8b40ef82b77db5031ee755: Log is configured to *not* fsync() on all Append() calls
I20260811 18:41:58.872062 19585 log.cc:824] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920: Log is configured to *not* fsync() on all Append() calls
I20260811 18:41:58.874162 19582 tablet_bootstrap.cc:492] T 0e9823a128bf4401bc0a4423e2a938aa P 5427cf526a8b40ef82b77db5031ee755: No bootstrap required, opened a new log
I20260811 18:41:58.874567 19582 ts_tablet_manager.cc:1397] T 0e9823a128bf4401bc0a4423e2a938aa P 5427cf526a8b40ef82b77db5031ee755: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.007s
I20260811 18:41:58.874862 19588 tablet_bootstrap.cc:492] T 0e9823a128bf4401bc0a4423e2a938aa P c5494fbb9d7d4e43b699f934eb6ba6c4: Bootstrap starting.
I20260811 18:41:58.875392 19585 tablet_bootstrap.cc:492] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920: No bootstrap required, opened a new log
I20260811 18:41:58.875720 19585 ts_tablet_manager.cc:1397] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260811 18:41:58.878321 19588 tablet_bootstrap.cc:654] T 0e9823a128bf4401bc0a4423e2a938aa P c5494fbb9d7d4e43b699f934eb6ba6c4: Neither blocks nor log segments found. Creating new log.
I20260811 18:41:58.879415 19588 log.cc:824] T 0e9823a128bf4401bc0a4423e2a938aa P c5494fbb9d7d4e43b699f934eb6ba6c4: Log is configured to *not* fsync() on all Append() calls
I20260811 18:41:58.882596 19588 tablet_bootstrap.cc:492] T 0e9823a128bf4401bc0a4423e2a938aa P c5494fbb9d7d4e43b699f934eb6ba6c4: No bootstrap required, opened a new log
I20260811 18:41:58.882979 19588 ts_tablet_manager.cc:1397] T 0e9823a128bf4401bc0a4423e2a938aa P c5494fbb9d7d4e43b699f934eb6ba6c4: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260811 18:41:58.886125 19582 raft_consensus.cc:348] T 0e9823a128bf4401bc0a4423e2a938aa P 5427cf526a8b40ef82b77db5031ee755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0703326c1ec04ec794bb9e3a03386920" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42899 } } peers { permanent_uuid: "c5494fbb9d7d4e43b699f934eb6ba6c4" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39633 } } peers { permanent_uuid: "5427cf526a8b40ef82b77db5031ee755" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38861 } }
I20260811 18:41:58.886709 19582 raft_consensus.cc:374] T 0e9823a128bf4401bc0a4423e2a938aa P 5427cf526a8b40ef82b77db5031ee755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:58.886577 19585 raft_consensus.cc:348] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0703326c1ec04ec794bb9e3a03386920" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42899 } } peers { permanent_uuid: "c5494fbb9d7d4e43b699f934eb6ba6c4" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39633 } } peers { permanent_uuid: "5427cf526a8b40ef82b77db5031ee755" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38861 } }
I20260811 18:41:58.887053 19585 raft_consensus.cc:374] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:58.887013 19582 raft_consensus.cc:729] T 0e9823a128bf4401bc0a4423e2a938aa P 5427cf526a8b40ef82b77db5031ee755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5427cf526a8b40ef82b77db5031ee755, State: Initialized, Role: FOLLOWER
I20260811 18:41:58.887171 19585 raft_consensus.cc:729] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0703326c1ec04ec794bb9e3a03386920, State: Initialized, Role: FOLLOWER
I20260811 18:41:58.888010 19582 consensus_queue.cc:260] T 0e9823a128bf4401bc0a4423e2a938aa P 5427cf526a8b40ef82b77db5031ee755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0703326c1ec04ec794bb9e3a03386920" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42899 } } peers { permanent_uuid: "c5494fbb9d7d4e43b699f934eb6ba6c4" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39633 } } peers { permanent_uuid: "5427cf526a8b40ef82b77db5031ee755" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38861 } }
I20260811 18:41:58.887989 19585 consensus_queue.cc:260] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0703326c1ec04ec794bb9e3a03386920" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42899 } } peers { permanent_uuid: "c5494fbb9d7d4e43b699f934eb6ba6c4" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39633 } } peers { permanent_uuid: "5427cf526a8b40ef82b77db5031ee755" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38861 } }
I20260811 18:41:58.889865 19548 heartbeater.cc:503] Master 127.30.185.190:41193 was elected leader, sending a full tablet report...
I20260811 18:41:58.890158 19582 ts_tablet_manager.cc:1428] T 0e9823a128bf4401bc0a4423e2a938aa P 5427cf526a8b40ef82b77db5031ee755: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
I20260811 18:41:58.890391 19585 ts_tablet_manager.cc:1428] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260811 18:41:58.893237 19588 raft_consensus.cc:348] T 0e9823a128bf4401bc0a4423e2a938aa P c5494fbb9d7d4e43b699f934eb6ba6c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0703326c1ec04ec794bb9e3a03386920" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42899 } } peers { permanent_uuid: "c5494fbb9d7d4e43b699f934eb6ba6c4" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39633 } } peers { permanent_uuid: "5427cf526a8b40ef82b77db5031ee755" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38861 } }
I20260811 18:41:58.893723 19588 raft_consensus.cc:374] T 0e9823a128bf4401bc0a4423e2a938aa P c5494fbb9d7d4e43b699f934eb6ba6c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:41:58.893833 19588 raft_consensus.cc:729] T 0e9823a128bf4401bc0a4423e2a938aa P c5494fbb9d7d4e43b699f934eb6ba6c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5494fbb9d7d4e43b699f934eb6ba6c4, State: Initialized, Role: FOLLOWER
I20260811 18:41:58.894623 19588 consensus_queue.cc:260] T 0e9823a128bf4401bc0a4423e2a938aa P c5494fbb9d7d4e43b699f934eb6ba6c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0703326c1ec04ec794bb9e3a03386920" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42899 } } peers { permanent_uuid: "c5494fbb9d7d4e43b699f934eb6ba6c4" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39633 } } peers { permanent_uuid: "5427cf526a8b40ef82b77db5031ee755" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38861 } }
I20260811 18:41:58.896450 19290 heartbeater.cc:503] Master 127.30.185.190:41193 was elected leader, sending a full tablet report...
I20260811 18:41:58.896660 19588 ts_tablet_manager.cc:1428] T 0e9823a128bf4401bc0a4423e2a938aa P c5494fbb9d7d4e43b699f934eb6ba6c4: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
W20260811 18:41:58.935333 19293 tablet.cc:2369] T 0e9823a128bf4401bc0a4423e2a938aa P c5494fbb9d7d4e43b699f934eb6ba6c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:41:58.941186 19590 raft_consensus.cc:482] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:41:58.941557 19590 raft_consensus.cc:504] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0703326c1ec04ec794bb9e3a03386920" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42899 } } peers { permanent_uuid: "c5494fbb9d7d4e43b699f934eb6ba6c4" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39633 } } peers { permanent_uuid: "5427cf526a8b40ef82b77db5031ee755" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38861 } }
I20260811 18:41:58.943679 19590 leader_election.cc:288] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5494fbb9d7d4e43b699f934eb6ba6c4 (127.30.185.130:39633), 5427cf526a8b40ef82b77db5031ee755 (127.30.185.129:38861)
I20260811 18:41:58.949429 19554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:58.943880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:39633 (local address 127.30.185.131:52345)
0811 18:41:58.944193 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:58.944216 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:58.944432 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:58.944838 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:58.944848 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:58.944889 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:58.945090 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:58.945104 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.946131 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.946137 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:58.947501 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:58.947513 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.948634 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.948640 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:58.948834 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:58.948860 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 18:41:58.948966 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 18:41:58.949102 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260811 18:41:58.949434 19581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:58.943986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:52345 (local address 127.30.185.130:39633)
0811 18:41:58.944192 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:58.944198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:58.944308 (+ 110us) server_negotiation.cc:407] Connection header received
0811 18:41:58.944629 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:58.944635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:58.944705 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:58.944805 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:58.945255 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.945987 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.947677 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.948464 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.948636 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:58.948712 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:58.949006 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 18:41:58.949141 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:41:58.950706 19578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:58.944326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:58349 (local address 127.30.185.129:38861)
0811 18:41:58.944568 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:58.944575 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:58.945120 (+ 545us) server_negotiation.cc:407] Connection header received
0811 18:41:58.945326 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:58.945330 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:58.945390 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:58.945490 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:41:58.946017 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.946983 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.948795 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.949808 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.949972 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:58.950057 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:58.950423 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 18:41:58.950525 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 18:41:58.950874 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:58.944167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:38861 (local address 127.30.185.131:58349)
0811 18:41:58.945006 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0811 18:41:58.945024 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:41:58.945133 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:41:58.945504 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:41:58.945508 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:41:58.945532 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:41:58.945850 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:58.945858 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.947131 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.947157 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:58.948630 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:41:58.948643 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.950028 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.950036 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:41:58.950227 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:58.950267 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 18:41:58.950385 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 18:41:58.950520 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":476,"threads_started":1}
I20260811 18:41:58.950654 19219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e9823a128bf4401bc0a4423e2a938aa" candidate_uuid: "0703326c1ec04ec794bb9e3a03386920" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5494fbb9d7d4e43b699f934eb6ba6c4" is_pre_election: true
I20260811 18:41:58.951328 19219 raft_consensus.cc:2454] T 0e9823a128bf4401bc0a4423e2a938aa P c5494fbb9d7d4e43b699f934eb6ba6c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0703326c1ec04ec794bb9e3a03386920 in term 0.
I20260811 18:41:58.952108 19338 leader_election.cc:302] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0703326c1ec04ec794bb9e3a03386920, c5494fbb9d7d4e43b699f934eb6ba6c4; no voters:
I20260811 18:41:58.952020 18978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e9823a128bf4401bc0a4423e2a938aa" candidate_uuid: "0703326c1ec04ec794bb9e3a03386920" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5427cf526a8b40ef82b77db5031ee755" is_pre_election: true
I20260811 18:41:58.952553 19590 raft_consensus.cc:2781] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:41:58.952526 18978 raft_consensus.cc:2454] T 0e9823a128bf4401bc0a4423e2a938aa P 5427cf526a8b40ef82b77db5031ee755 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0703326c1ec04ec794bb9e3a03386920 in term 0.
I20260811 18:41:58.952716 19590 raft_consensus.cc:482] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:41:58.952834 19590 raft_consensus.cc:3037] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:58.955255 19590 raft_consensus.cc:504] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0703326c1ec04ec794bb9e3a03386920" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42899 } } peers { permanent_uuid: "c5494fbb9d7d4e43b699f934eb6ba6c4" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39633 } } peers { permanent_uuid: "5427cf526a8b40ef82b77db5031ee755" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38861 } }
I20260811 18:41:58.955972 19590 leader_election.cc:288] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920 [CANDIDATE]: Term 1 election: Requested vote from peers c5494fbb9d7d4e43b699f934eb6ba6c4 (127.30.185.130:39633), 5427cf526a8b40ef82b77db5031ee755 (127.30.185.129:38861)
I20260811 18:41:58.956458 19219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e9823a128bf4401bc0a4423e2a938aa" candidate_uuid: "0703326c1ec04ec794bb9e3a03386920" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5494fbb9d7d4e43b699f934eb6ba6c4"
I20260811 18:41:58.956569 18978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e9823a128bf4401bc0a4423e2a938aa" candidate_uuid: "0703326c1ec04ec794bb9e3a03386920" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5427cf526a8b40ef82b77db5031ee755"
I20260811 18:41:58.956679 19219 raft_consensus.cc:3037] T 0e9823a128bf4401bc0a4423e2a938aa P c5494fbb9d7d4e43b699f934eb6ba6c4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:58.956825 18978 raft_consensus.cc:3037] T 0e9823a128bf4401bc0a4423e2a938aa P 5427cf526a8b40ef82b77db5031ee755 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:41:58.959129 19219 raft_consensus.cc:2454] T 0e9823a128bf4401bc0a4423e2a938aa P c5494fbb9d7d4e43b699f934eb6ba6c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0703326c1ec04ec794bb9e3a03386920 in term 1.
I20260811 18:41:58.959228 18978 raft_consensus.cc:2454] T 0e9823a128bf4401bc0a4423e2a938aa P 5427cf526a8b40ef82b77db5031ee755 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0703326c1ec04ec794bb9e3a03386920 in term 1.
I20260811 18:41:58.959640 19338 leader_election.cc:302] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0703326c1ec04ec794bb9e3a03386920, c5494fbb9d7d4e43b699f934eb6ba6c4; no voters:
I20260811 18:41:58.959978 19590 raft_consensus.cc:2781] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:41:58.960424 19590 raft_consensus.cc:686] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920 [term 1 LEADER]: Becoming Leader. State: Replica: 0703326c1ec04ec794bb9e3a03386920, State: Running, Role: LEADER
I20260811 18:41:58.960929 19590 consensus_queue.cc:237] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0703326c1ec04ec794bb9e3a03386920" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42899 } } peers { permanent_uuid: "c5494fbb9d7d4e43b699f934eb6ba6c4" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39633 } } peers { permanent_uuid: "5427cf526a8b40ef82b77db5031ee755" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38861 } }
I20260811 18:41:58.966606 18707 catalog_manager.cc:5495] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0703326c1ec04ec794bb9e3a03386920 (127.30.185.131). New cstate: current_term: 1 leader_uuid: "0703326c1ec04ec794bb9e3a03386920" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0703326c1ec04ec794bb9e3a03386920" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5494fbb9d7d4e43b699f934eb6ba6c4" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5427cf526a8b40ef82b77db5031ee755" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38861 } health_report { overall_health: UNKNOWN } } }
I20260811 18:41:58.985365 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:58.980143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51294 (local address 127.30.185.131:42899)
0811 18:41:58.980339 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:58.980343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:58.980361 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:41:58.980542 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:58.980545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:58.980636 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:58.980759 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:41:58.981221 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.981965 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.983278 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:58.983677 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:58.983769 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:58.984211 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:41:58.984732 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:41:58.984843 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:58.984950 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 18:41:58.985019 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 18:41:58.991091 19549 tablet.cc:2369] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:41:58.994148 19219 raft_consensus.cc:1261] T 0e9823a128bf4401bc0a4423e2a938aa P c5494fbb9d7d4e43b699f934eb6ba6c4 [term 1 FOLLOWER]: Refusing update from remote peer 0703326c1ec04ec794bb9e3a03386920: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:41:58.994154 18978 raft_consensus.cc:1261] T 0e9823a128bf4401bc0a4423e2a938aa P 5427cf526a8b40ef82b77db5031ee755 [term 1 FOLLOWER]: Refusing update from remote peer 0703326c1ec04ec794bb9e3a03386920: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:41:58.995221 19590 consensus_queue.cc:1035] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5494fbb9d7d4e43b699f934eb6ba6c4" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:41:58.995543 19595 consensus_queue.cc:1035] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5427cf526a8b40ef82b77db5031ee755" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:41:59.022408 18708 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:41:59.022717 18708 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:41:59.026633 18708 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b84e8fc033d541e8b492c74ebcf1fe9e: Sending DeleteTablet for 3 replicas of tablet 0e9823a128bf4401bc0a4423e2a938aa
I20260811 18:41:59.028007 18943 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e9823a128bf4401bc0a4423e2a938aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:41:59 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:34194
I20260811 18:41:59.028014 19177 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e9823a128bf4401bc0a4423e2a938aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:41:59 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:54526
I20260811 18:41:59.028251 19418 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e9823a128bf4401bc0a4423e2a938aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:41:59 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:51292
I20260811 18:41:59.029433 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 18823
I20260811 18:41:59.033007 19619 tablet_replica.cc:330] T 0e9823a128bf4401bc0a4423e2a938aa P 5427cf526a8b40ef82b77db5031ee755: stopping tablet replica
I20260811 18:41:59.033566 19619 raft_consensus.cc:2229] T 0e9823a128bf4401bc0a4423e2a938aa P 5427cf526a8b40ef82b77db5031ee755 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:41:59.034235 19619 raft_consensus.cc:2258] T 0e9823a128bf4401bc0a4423e2a938aa P 5427cf526a8b40ef82b77db5031ee755 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:41:59.036835 19619 ts_tablet_manager.cc:1905] T 0e9823a128bf4401bc0a4423e2a938aa P 5427cf526a8b40ef82b77db5031ee755: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 18:41:59 dist-test-slave-z8x9 krb5kdc[18622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.129@KRBTEST.COM
I20260811 18:41:59.045328 19619 ts_tablet_manager.cc:1918] T 0e9823a128bf4401bc0a4423e2a938aa P 5427cf526a8b40ef82b77db5031ee755: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:41:59.045639 19619 log.cc:1196] T 0e9823a128bf4401bc0a4423e2a938aa P 5427cf526a8b40ef82b77db5031ee755: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-0/wal/wals/0e9823a128bf4401bc0a4423e2a938aa
I20260811 18:41:59.046411 19619 ts_tablet_manager.cc:1939] T 0e9823a128bf4401bc0a4423e2a938aa P 5427cf526a8b40ef82b77db5031ee755: Deleting consensus metadata
I20260811 18:41:59.048867 19621 tablet_replica.cc:330] T 0e9823a128bf4401bc0a4423e2a938aa P c5494fbb9d7d4e43b699f934eb6ba6c4: stopping tablet replica
I20260811 18:41:59.049000 18679 catalog_manager.cc:4841] TS 5427cf526a8b40ef82b77db5031ee755 (127.30.185.129:38861): tablet 0e9823a128bf4401bc0a4423e2a938aa (table test-table [id=cc97b894de9a4164bd8fa115c1af3c36]) successfully deleted
I20260811 18:41:59.049470 19621 raft_consensus.cc:2229] T 0e9823a128bf4401bc0a4423e2a938aa P c5494fbb9d7d4e43b699f934eb6ba6c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:41:59.049950 19621 raft_consensus.cc:2258] T 0e9823a128bf4401bc0a4423e2a938aa P c5494fbb9d7d4e43b699f934eb6ba6c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:41:59.054205 19578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:59.030755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34206 (local address 127.30.185.129:38861)
0811 18:41:59.030951 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:59.030957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:59.032348 (+ 1391us) server_negotiation.cc:407] Connection header received
0811 18:41:59.032541 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:59.032549 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:59.032652 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:59.032772 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:59.034964 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:59.035744 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:59.037026 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:59.037910 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:59.038077 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:59.045216 (+ 7139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:59.045239 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:59.045266 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:59.045344 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:59.050413 (+ 5069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:59.051366 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:59.051379 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:59.051386 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:59.051607 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:59.052078 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:59.052086 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:59.052092 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:59.052680 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:59.053088 (+ 408us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:59.053431 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 18:41:59.053748 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 18:41:59.055002 19049 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:59.056430 19622 tablet_replica.cc:330] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920: stopping tablet replica
I20260811 18:41:59.057729 19622 raft_consensus.cc:2229] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:41:59.058473 19622 raft_consensus.cc:2258] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:41:59.058844 19621 ts_tablet_manager.cc:1905] T 0e9823a128bf4401bc0a4423e2a938aa P c5494fbb9d7d4e43b699f934eb6ba6c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:41:59.061218 19622 ts_tablet_manager.cc:1905] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:41:59.066066 19621 ts_tablet_manager.cc:1918] T 0e9823a128bf4401bc0a4423e2a938aa P c5494fbb9d7d4e43b699f934eb6ba6c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:41:59.066342 19621 log.cc:1196] T 0e9823a128bf4401bc0a4423e2a938aa P c5494fbb9d7d4e43b699f934eb6ba6c4: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/0e9823a128bf4401bc0a4423e2a938aa
I20260811 18:41:59.067030 19621 ts_tablet_manager.cc:1939] T 0e9823a128bf4401bc0a4423e2a938aa P c5494fbb9d7d4e43b699f934eb6ba6c4: Deleting consensus metadata
I20260811 18:41:59.069108 18678 catalog_manager.cc:4841] TS c5494fbb9d7d4e43b699f934eb6ba6c4 (127.30.185.130:39633): tablet 0e9823a128bf4401bc0a4423e2a938aa (table test-table [id=cc97b894de9a4164bd8fa115c1af3c36]) successfully deleted
I20260811 18:41:59.071305 19622 ts_tablet_manager.cc:1918] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:41:59.071513 19622 log.cc:1196] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/0e9823a128bf4401bc0a4423e2a938aa
I20260811 18:41:59.072010 19622 ts_tablet_manager.cc:1939] T 0e9823a128bf4401bc0a4423e2a938aa P 0703326c1ec04ec794bb9e3a03386920: Deleting consensus metadata
I20260811 18:41:59.073546 18678 catalog_manager.cc:4841] TS 0703326c1ec04ec794bb9e3a03386920 (127.30.185.131:42899): tablet 0e9823a128bf4401bc0a4423e2a938aa (table test-table [id=cc97b894de9a4164bd8fa115c1af3c36]) successfully deleted
I20260811 18:41:59.194427 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 18823
I20260811 18:41:59.225920 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 19069
Aug 11 18:41:59 dist-test-slave-z8x9 krb5kdc[18622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.130@KRBTEST.COM
I20260811 18:41:59.242589 19581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:59.226805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54542 (local address 127.30.185.130:39633)
0811 18:41:59.227013 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:59.227020 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:59.227043 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:41:59.227232 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:59.227238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:59.227337 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:59.227444 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:59.228615 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:59.229982 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:59.231235 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:59.231845 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:59.231976 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:59.236553 (+ 4577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:59.236565 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:59.236580 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:59.236620 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:59.239520 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:59.240388 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:59.240394 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:59.240397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:59.240522 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:59.240931 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:59.240935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:59.240937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:59.241401 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:59.241623 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:59.242081 (+ 458us) server_negotiation.cc:299] Negotiation successful
0811 18:41:59.242251 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 18:41:59.243324 19276 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:59.334724 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 19069
I20260811 18:41:59.359001 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 19301
Aug 11 18:41:59 dist-test-slave-z8x9 krb5kdc[18622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.131@KRBTEST.COM
I20260811 18:41:59.377424 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:41:59.359933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51304 (local address 127.30.185.131:42899)
0811 18:41:59.360110 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 18:41:59.360115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:41:59.360207 (+ 92us) server_negotiation.cc:407] Connection header received
0811 18:41:59.360434 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:41:59.360439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:41:59.360521 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:41:59.360598 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:41:59.362546 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:59.363353 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:59.365329 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:41:59.366027 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:41:59.366160 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:41:59.370888 (+ 4728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:41:59.370908 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:41:59.370930 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:41:59.370977 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:41:59.374265 (+ 3288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:59.375099 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:59.375106 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:59.375108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:59.375281 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:41:59.375741 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:41:59.375745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:41:59.375747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:41:59.376113 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:41:59.376364 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 18:41:59.376737 (+ 373us) server_negotiation.cc:299] Negotiation successful
0811 18:41:59.376938 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 18:41:59.378257 19532 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:59.505556 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 19301
I20260811 18:41:59.528831 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 18647
I20260811 18:41:59.530160 18764 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:41:59.641009 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 18647
2026-08-11T18:41:59Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4908 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:41:59 dist-test-slave-z8x9 krb5kdc[19637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:41:59 dist-test-slave-z8x9 krb5kdc[19637](info): set up 2 sockets
Aug 11 18:41:59 dist-test-slave-z8x9 krb5kdc[19637](info): commencing operation
krb5kdc: starting...
W20260811 18:42:01.061975 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.371s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:42:01 dist-test-slave-z8x9 krb5kdc[19637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:42:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:42:01Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.185.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.185.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.185.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/rangeradmin_127.30.185.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.185.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/rangeradmin_127.30.185.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.185.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.185.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.185.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/rangerlookup_127.30.185.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.185.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/rangerlookup_127.30.185.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.185.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.185.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.185.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/HTTP_127.30.185.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.185.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/HTTP_127.30.185.148@KRBTEST.COM.keytab.
I20260811 18:42:01.516088 31462 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasky46AhN/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/postgres -l logfile start
2026-08-11 18:42:04.199 UTC [19702] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 18:42:04.199 UTC [19702] LOG: listening on IPv4 address "127.30.185.148", port 39871
2026-08-11 18:42:04.203 UTC [19702] LOG: listening on Unix socket "/tmp/.s.PGSQL.39871"
2026-08-11 18:42:04.232 UTC [19706] LOG: database system was shut down at 2026-08-11 18:42:02 UTC
2026-08-11 18:42:04.237 UTC [19702] LOG: database system is ready to accept connections
I20260811 18:42:05.539479 31462 mini_postgres.cc:96] Postgres bound to 39871
2026-08-11 18:42:05.567 UTC [19731] FATAL: database "slave" does not exist
127.30.185.148:39871 - accepting connections
I20260811 18:42:05.568792 31462 mini_ranger.cc:161] Starting Ranger...
I20260811 18:42:05.604374 31462 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 18:42:06.116808 31462 mini_ranger.cc:87] Created ranger Postgres database
I20260811 18:42:06.117002 31462 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ranger-admin
2026-08-11 18:42:06,511 [I] DB FLAVOR :POSTGRES
2026-08-11 18:42:06,511 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 18:42:06,512 [I] Checking connection..
2026-08-11 18:42:06,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:39871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 18:42:06,776 [I] Checking connection passed.
2026-08-11 18:42:06,776 [I] --------- Verifying version history table ---------
2026-08-11 18:42:06,776 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:39871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:42:07,037 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 18:42:07,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:39871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:42:07,309 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 18:42:07,309 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 18:42:07,309 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:39871/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 18:42:07.577 UTC [19852] WARNING: there is no transaction in progress
2026-08-11 18:42:07,587 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 18:42:07,588 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:39871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:42:07,863 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 18:42:07,863 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 18:42:07,864 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:39871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 18:42:08,104 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:39871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 18:42:08,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:39871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-z8x9.c.gcp-upstream.internal','N') ;"
2026-08-11 18:42:08,628 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:39871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 18:42:08,905 [I] Table x_portal_user does not exist in database ranger
2026-08-11 18:42:08,905 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:39871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 18:42:09,164 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 18:42:09,164 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:39871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 18:42:09,434 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:39871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 18:42:09,705 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 18:42:09,706 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:39871/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 18:42:10.723 UTC [20055] WARNING: there is no transaction in progress
2026-08-11 18:42:10.739 UTC [20055] WARNING: there is no transaction in progress
2026-08-11 18:42:10.756 UTC [20055] WARNING: there is no transaction in progress
2026-08-11 18:42:10.771 UTC [20055] WARNING: there is no transaction in progress
2026-08-11 18:42:10.791 UTC [20055] WARNING: there is no transaction in progress
2026-08-11 18:42:10.808 UTC [20055] WARNING: there is no transaction in progress
2026-08-11 18:42:10.868 UTC [20055] WARNING: there is no transaction in progress
2026-08-11 18:42:10.874 UTC [20055] WARNING: there is no transaction in progress
2026-08-11 18:42:10.888 UTC [20055] WARNING: there is no transaction in progress
2026-08-11 18:42:10.898 UTC [20055] WARNING: there is no transaction in progress
2026-08-11 18:42:10.909 UTC [20055] WARNING: there is no transaction in progress
2026-08-11 18:42:10.924 UTC [20055] WARNING: there is no transaction in progress
2026-08-11 18:42:10.934 UTC [20055] WARNING: there is no transaction in progress
2026-08-11 18:42:10.941 UTC [20055] WARNING: there is no transaction in progress
2026-08-11 18:42:10.948 UTC [20055] WARNING: there is no transaction in progress
2026-08-11 18:42:10.954 UTC [20055] WARNING: there is no transaction in progress
2026-08-11 18:42:11,612 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 18:42:11,613 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:39871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 18:42:11,857 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 18:42:11,857 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:39871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 18:42:12,121 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 18:42:12,121 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:39871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 18:42:12,384 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 18:42:12,384 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:39871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 18:42:12,648 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:39871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 18:42:12,907 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:39871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-z8x9.c.gcp-upstream.internal';"
2026-08-11 18:42:13,169 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 18:42:13,169 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:39871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 18:42:13,441 [I] DB_PATCHES have already been applied
I20260811 18:42:13.447932 31462 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ranger-admin:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasky46AhN/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 18:42:13.448062 31462 mini_ranger.cc:193] Using host: 127.30.185.148
I20260811 18:42:13.472890 31462 mini_ranger.cc:227] Ranger admin URL: http://127.30.185.148:49545
Aug 11, 2026 6:42:13 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:42:13 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:42:14 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 6:42:14 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 6:42:14 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 6:42:14 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 6:42:14 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 18:42:14.971598 31462 mini_ranger.cc:160] Time spent starting Ranger: real 9.403s user 0.010s sys 0.069s
Aug 11, 2026 6:42:35 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 6:42:35 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 11, 2026 6:42:35 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 6:42:35 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 6:42:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 6:42:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 6:42:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 6:42:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 6:42:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 6:42:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 6:42:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 6:42:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 6:42:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 6:42:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 6:42:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 6:42:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 6:42:35 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 6:42:36 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 18:42:37.093309 31462 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.185.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.185.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.185.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/rangerkms_127.30.185.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.185.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/rangerkms_127.30.185.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.185.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.185.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.185.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/HTTP_127.30.185.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.185.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/HTTP_127.30.185.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 18:42:37 dist-test-slave-z8x9 krb5kdc[19637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473757, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 18:42:37.339150 31462 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 18:42:37.371366 31462 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 18:42:38.274590 31462 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 18:42:38.274713 31462 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ranger-kms
I20260811 18:42:38.274936 31462 mini_ranger_kms.cc:227] Using postgres at 127.30.185.148:39871
2026-08-11 18:42:38,797 [I] DB FLAVOR :POSTGRES
2026-08-11 18:42:38,797 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 18:42:38,797 [I] Checking connection
2026-08-11 18:42:38,797 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:39871/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 18:42:39,053 [I] connection success
2026-08-11 18:42:39,054 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 18:42:39,054 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 18:42:39,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:39871/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 18:42:39,293 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 18:42:39,293 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 18:42:39,293 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 18:42:39,293 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:39871/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 18:42:39,587 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 18:42:40.053126 31462 mini_ranger_kms.cc:310] Created kms service
I20260811 18:42:40.264401 31462 mini_ranger_kms.cc:326] Created kudu user
I20260811 18:42:40.319751 31462 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 18:42:40 dist-test-slave-z8x9 krb5kdc[19637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473757, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
I20260811 18:42:40.776782 31462 mini_ranger_kms.cc:384] Added ranger policy
I20260811 18:42:40.778152 31462 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ranger-kms:/tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 18:42:40.778260 31462 mini_ranger_kms.cc:240] Using host: 127.30.185.148
I20260811 18:42:40.813658 31462 mini_ranger_kms.cc:276] Ranger KMS PID: 20403
I20260811 18:42:40.813910 31462 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.185.148:47575
Aug 11, 2026 6:42:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:42:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:42:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 6:42:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 6:42:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 6:42:42 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 18:42:43 dist-test-slave-z8x9 krb5kdc[19637](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786473763, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.185.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 18:42:43 dist-test-slave-z8x9 krb5kdc[19637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473763, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.185.148@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
I20260811 18:42:43.869428 31462 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.530s user 0.135s sys 0.444s
I20260811 18:42:43.869807 31462 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 18:42:43.870054 31462 mini_ranger_kms.cc:401] 127.30.185.148:47575/kms/v1/keys
Aug 11 18:42:44 dist-test-slave-z8x9 krb5kdc[19637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473763, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.185.148@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
Aug 11 18:42:46 dist-test-slave-z8x9 krb5kdc[19637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:42:46.505595 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:39639
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:33305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:39639
--ranger_config_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.185.148:47575/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:42:46.954107 20495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:42:46.954443 20495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:42:46.954538 20495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:42:46.962177 20495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:42:46.962325 20495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:42:46.962375 20495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:42:46.962437 20495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:42:46.962584 20495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:42:46.974658 20495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:33305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.185.148:47575/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:39639
--ranger_config_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:39639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:42:46.976258 20495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:42:46.977895 20495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:42:46.991333 20511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:42:46.991554 20509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:42:46.991596 20505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:42:46.992357 20495 server_base.cc:1034] running on GCE node
I20260811 18:42:46.993244 20495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:42:46.994710 20495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:42:46.995972 20495 hybrid_clock.cc:648] HybridClock initialized: now 1786473766995919 us; error 45 us; skew 500 ppm
I20260811 18:42:46.996411 20495 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:42:46 dist-test-slave-z8x9 krb5kdc[19637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473766, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:42:47.000244 20495 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:42:47.002033 20495 webserver.cc:460] Webserver started at http://127.30.185.190:35659/ using document root <none> and password file <none>
I20260811 18:42:47.002676 20495 fs_manager.cc:359] Metadata directory not provided
I20260811 18:42:47.002780 20495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:42:47.003036 20495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:42:47 dist-test-slave-z8x9 krb5kdc[19637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473766, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
I20260811 18:42:47.073127 20495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "a1aa9bed12924aab8fa1615d9ab3923b"
format_stamp: "Formatted at 2026-08-11 18:42:47 on dist-test-slave-z8x9"
server_key: "53b3bde9dcadaf3058c4724540e53ff5"
server_key_iv: "d1cafe14cd428453e63c0a71c27c1cf8"
server_key_version: "kuduclusterkey@0"
I20260811 18:42:47.074105 20495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "a1aa9bed12924aab8fa1615d9ab3923b"
format_stamp: "Formatted at 2026-08-11 18:42:47 on dist-test-slave-z8x9"
server_key: "53b3bde9dcadaf3058c4724540e53ff5"
server_key_iv: "d1cafe14cd428453e63c0a71c27c1cf8"
server_key_version: "kuduclusterkey@0"
I20260811 18:42:47.080644 20495 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260811 18:42:47.101521 20526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:42:47.103150 20495 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260811 18:42:47.103363 20495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "a1aa9bed12924aab8fa1615d9ab3923b"
format_stamp: "Formatted at 2026-08-11 18:42:47 on dist-test-slave-z8x9"
server_key: "53b3bde9dcadaf3058c4724540e53ff5"
server_key_iv: "d1cafe14cd428453e63c0a71c27c1cf8"
server_key_version: "kuduclusterkey@0"
I20260811 18:42:47.103617 20495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:42:47.131935 20495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:42:47.136533 20495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:42:47.136909 20495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:42:47.166109 20495 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:39639
I20260811 18:42:47.166289 20616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:39639 every 8 connection(s)
I20260811 18:42:47.167662 20495 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:42:47.168906 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 20495
I20260811 18:42:47.169153 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:42:47.170617 31462 external_mini_cluster.cc:1428] Setting key 799997c3f687851a72ee586f6acf15df
I20260811 18:42:47.173223 20622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:42:47.191193 20622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b: Bootstrap starting.
I20260811 18:42:47.195829 20622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b: Neither blocks nor log segments found. Creating new log.
I20260811 18:42:47.197196 20622 log.cc:824] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:42:47 dist-test-slave-z8x9 krb5kdc[19637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:42:47.200413 20622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b: No bootstrap required, opened a new log
I20260811 18:42:47.210420 20627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:42:47.174197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49490 (local address 127.30.185.190:39639)
0811 18:42:47.184724 (+ 10527us) server_negotiation.cc:206] Beginning negotiation
0811 18:42:47.184749 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 18:42:47.184824 (+ 75us) server_negotiation.cc:407] Connection header received
0811 18:42:47.188072 (+ 3248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:42:47.188100 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:42:47.188528 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:42:47.188963 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:42:47.190517 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:47.192125 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:47.193492 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:47.194169 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:47.194369 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:42:47.198935 (+ 4566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:42:47.198947 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:42:47.198968 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:42:47.199038 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:42:47.204147 (+ 5109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:42:47.205091 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:42:47.205101 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:42:47.205114 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:42:47.205441 (+ 327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:42:47.205912 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:42:47.205918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:42:47.205921 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:42:47.206909 (+ 988us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:42:47.207307 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0811 18:42:47.208141 (+ 834us) server_negotiation.cc:299] Negotiation successful
0811 18:42:47.208651 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10106,"thread_start_us":10832,"threads_started":1}
I20260811 18:42:47.211934 20622 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1aa9bed12924aab8fa1615d9ab3923b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39639 } }
I20260811 18:42:47.212451 20622 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:42:47.212617 20622 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1aa9bed12924aab8fa1615d9ab3923b, State: Initialized, Role: FOLLOWER
I20260811 18:42:47.213390 20622 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1aa9bed12924aab8fa1615d9ab3923b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39639 } }
I20260811 18:42:47.213712 20622 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:42:47.213950 20622 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:42:47.214275 20622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:42:47.216897 20622 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1aa9bed12924aab8fa1615d9ab3923b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39639 } }
I20260811 18:42:47.217463 20622 leader_election.cc:302] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1aa9bed12924aab8fa1615d9ab3923b; no voters:
I20260811 18:42:47.218281 20622 leader_election.cc:288] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:42:47.218400 20630 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:42:47.220150 20630 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b [term 1 LEADER]: Becoming Leader. State: Replica: a1aa9bed12924aab8fa1615d9ab3923b, State: Running, Role: LEADER
I20260811 18:42:47.220851 20630 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1aa9bed12924aab8fa1615d9ab3923b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39639 } }
I20260811 18:42:47.221105 20622 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:42:47.224084 20634 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1aa9bed12924aab8fa1615d9ab3923b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1aa9bed12924aab8fa1615d9ab3923b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39639 } } }
I20260811 18:42:47.224555 20634 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b [sys.catalog]: This master's current role is: LEADER
I20260811 18:42:47.225060 20635 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1aa9bed12924aab8fa1615d9ab3923b. Latest consensus state: current_term: 1 leader_uuid: "a1aa9bed12924aab8fa1615d9ab3923b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1aa9bed12924aab8fa1615d9ab3923b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39639 } } }
I20260811 18:42:47.225468 20635 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b [sys.catalog]: This master's current role is: LEADER
I20260811 18:42:47.233760 20643 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:42:47.241736 20643 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:42:47.242522 20622 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 18:42:47.262037 20643 catalog_manager.cc:1347] Generated new cluster ID: 4c3f770910044b56aa01c824d56e5f15
I20260811 18:42:47.262264 20643 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:42:47.300606 20643 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:42:47.302196 20643 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:42:47.308002 20643 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b: Generated new TSK 0
I20260811 18:42:47.308976 20643 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.185.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
I20260811 18:42:48.214044 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.185.148:47575/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:39639
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:33305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:42:48 dist-test-slave-z8x9 krb5kdc[19637](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786473768, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 18:42:48.752121 20712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:42:48.752444 20712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:42:48.752643 20712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:42:48.761157 20712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:42:48.761430 20712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:42:48.761631 20712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:42:48.772748 20712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:33305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.185.148:47575/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:39639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:42:48.774204 20712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:42:48.775583 20712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:42:48.812947 20735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:42:48.813875 20736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:42:48.819736 20743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:42:48.820936 20712 server_base.cc:1034] running on GCE node
I20260811 18:42:48.821877 20712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:42:48.825646 20712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:42:48.827011 20712 hybrid_clock.cc:648] HybridClock initialized: now 1786473768826833 us; error 117 us; skew 500 ppm
I20260811 18:42:48.827642 20712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:42:48 dist-test-slave-z8x9 krb5kdc[19637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473768, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:42:48.832118 20712 init.cc:377] Logged in from keytab as kudu/127.30.185.129@KRBTEST.COM (short username kudu)
I20260811 18:42:48.842900 20712 webserver.cc:460] Webserver started at http://127.30.185.129:33443/ using document root <none> and password file <none>
I20260811 18:42:48.843750 20712 fs_manager.cc:359] Metadata directory not provided
I20260811 18:42:48.843909 20712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:42:48.844262 20712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:42:48 dist-test-slave-z8x9 krb5kdc[19637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473768, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
I20260811 18:42:48.870369 20712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "6737218bdfec48958ba94ab64a85da51"
format_stamp: "Formatted at 2026-08-11 18:42:48 on dist-test-slave-z8x9"
server_key: "a8ff2662133bd40936994272dbac4d1d"
server_key_iv: "63d65e9870c0caf10b15876219c4b0e1"
server_key_version: "kuduclusterkey@0"
I20260811 18:42:48.871728 20712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "6737218bdfec48958ba94ab64a85da51"
format_stamp: "Formatted at 2026-08-11 18:42:48 on dist-test-slave-z8x9"
server_key: "a8ff2662133bd40936994272dbac4d1d"
server_key_iv: "63d65e9870c0caf10b15876219c4b0e1"
server_key_version: "kuduclusterkey@0"
I20260811 18:42:48.886797 20712 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.000s sys 0.012s
I20260811 18:42:48.920651 20755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:42:48.922894 20712 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.003s sys 0.006s
I20260811 18:42:48.923092 20712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "6737218bdfec48958ba94ab64a85da51"
format_stamp: "Formatted at 2026-08-11 18:42:48 on dist-test-slave-z8x9"
server_key: "a8ff2662133bd40936994272dbac4d1d"
server_key_iv: "63d65e9870c0caf10b15876219c4b0e1"
server_key_version: "kuduclusterkey@0"
I20260811 18:42:48.923344 20712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:42:48.955685 20712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:42:48.962523 20712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:42:48.963080 20712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:42:48.975903 20712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
Aug 11 18:42:48 dist-test-slave-z8x9 krb5kdc[19637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473768, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
I20260811 18:42:48.992944 20712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:42:48.993114 20712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:42:48.993345 20712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:42:48.993458 20712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:42:49.163885 20712 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:40227
I20260811 18:42:49.164215 21009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:40227 every 8 connection(s)
I20260811 18:42:49.165881 20712 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:42:49.174892 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 20712
I20260811 18:42:49.175150 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:42:49.176277 31462 external_mini_cluster.cc:1428] Setting key 82d50c483911fe231cb36858f1866737
Aug 11 18:42:49 dist-test-slave-z8x9 krb5kdc[19637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473768, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:42:49.210532 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:42:49.178351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:59083 (local address 127.30.185.190:39639)
0811 18:42:49.185812 (+ 7461us) server_negotiation.cc:206] Beginning negotiation
0811 18:42:49.185819 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:42:49.189734 (+ 3915us) server_negotiation.cc:407] Connection header received
0811 18:42:49.189804 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:42:49.189809 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:42:49.189887 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:42:49.189985 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:42:49.192139 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:49.193291 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:49.196397 (+ 3106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:49.197089 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:49.197256 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:42:49.202673 (+ 5417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:42:49.202681 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:42:49.202686 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:42:49.202727 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:42:49.206559 (+ 3832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:42:49.207544 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:42:49.207550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:42:49.207553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:42:49.207726 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:42:49.208286 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:42:49.208291 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:42:49.208294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:42:49.208662 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:42:49.208779 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 18:42:49.209856 (+ 1077us) server_negotiation.cc:299] Negotiation successful
0811 18:42:49.210101 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7329,"thread_start_us":7145,"threads_started":1}
I20260811 18:42:49.213011 21013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:42:49.173706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39639 (local address 127.30.185.129:59083)
0811 18:42:49.186475 (+ 12769us) negotiation.cc:107] Waiting for socket to connect
0811 18:42:49.186541 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 18:42:49.189344 (+ 2803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:42:49.190432 (+ 1088us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:42:49.190444 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:42:49.191121 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:42:49.191941 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:42:49.191966 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:42:49.193491 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:42:49.193496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:42:49.196200 (+ 2704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:42:49.196251 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:42:49.197725 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:42:49.197732 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:42:49.198003 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:42:49.198874 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:42:49.198942 (+ 68us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:42:49.202356 (+ 3414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:42:49.206767 (+ 4411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:42:49.206776 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:42:49.206795 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:42:49.207399 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:42:49.207889 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:42:49.207894 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:42:49.207897 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:42:49.208155 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:42:49.208890 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:42:49.208899 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:42:49.209407 (+ 508us) client_negotiation.cc:769] Sending connection context
0811 18:42:49.211084 (+ 1677us) client_negotiation.cc:240] Negotiation successful
0811 18:42:49.211653 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12446,"thread_start_us":12194,"threads_started":1}
I20260811 18:42:49.215516 21011 heartbeater.cc:347] Connected to a master server at 127.30.185.190:39639
I20260811 18:42:49.215866 21011 heartbeater.cc:464] Registering TS with master...
I20260811 18:42:49.217025 21011 heartbeater.cc:511] Master 127.30.185.190:39639 requested a full tablet report, sending...
I20260811 18:42:49.220170 20559 ts_manager.cc:194] Registered new tserver with Master: 6737218bdfec48958ba94ab64a85da51 (127.30.185.129:40227)
WARNING: no policy specified for kudu/127.30.185.130@KRBTEST.COM; defaulting to no policy
I20260811 18:42:49.222337 20559 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:59083
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
I20260811 18:42:49.270556 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.185.148:47575/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:39639
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:33305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:42:49 dist-test-slave-z8x9 krb5kdc[19637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473768, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
W20260811 18:42:49.824221 21024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:42:49.824523 21024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:42:49.824669 21024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:42:49.833416 21024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:42:49.833559 21024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:42:49.833825 21024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:42:49.844933 21024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:33305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.185.148:47575/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:39639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:42:49.846562 21024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:42:49.847996 21024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:42:49.863371 21052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:42:49.863960 21051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:42:49.864547 21024 server_base.cc:1034] running on GCE node
W20260811 18:42:49.864743 21057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:42:49.865563 21024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:42:49.866801 21024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:42:49.868043 21024 hybrid_clock.cc:648] HybridClock initialized: now 1786473769867975 us; error 60 us; skew 500 ppm
I20260811 18:42:49.868466 21024 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:42:49 dist-test-slave-z8x9 krb5kdc[19637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473769, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:42:49.872020 21024 init.cc:377] Logged in from keytab as kudu/127.30.185.130@KRBTEST.COM (short username kudu)
I20260811 18:42:49.873875 21024 webserver.cc:460] Webserver started at http://127.30.185.130:35329/ using document root <none> and password file <none>
I20260811 18:42:49.874593 21024 fs_manager.cc:359] Metadata directory not provided
I20260811 18:42:49.874794 21024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:42:49.875192 21024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:42:49 dist-test-slave-z8x9 krb5kdc[19637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473769, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
I20260811 18:42:49.893678 21024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "f50641a15d544c4e93b99496c5e91d2c"
format_stamp: "Formatted at 2026-08-11 18:42:49 on dist-test-slave-z8x9"
server_key: "9814b859e4b6f32188fc3e89f77ea124"
server_key_iv: "af80bf0da6ee29896853573936c34f57"
server_key_version: "kuduclusterkey@0"
I20260811 18:42:49.895064 21024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "f50641a15d544c4e93b99496c5e91d2c"
format_stamp: "Formatted at 2026-08-11 18:42:49 on dist-test-slave-z8x9"
server_key: "9814b859e4b6f32188fc3e89f77ea124"
server_key_iv: "af80bf0da6ee29896853573936c34f57"
server_key_version: "kuduclusterkey@0"
I20260811 18:42:49.901681 21024 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260811 18:42:49.921597 21079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:42:49.923520 21024 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 18:42:49.923804 21024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "f50641a15d544c4e93b99496c5e91d2c"
format_stamp: "Formatted at 2026-08-11 18:42:49 on dist-test-slave-z8x9"
server_key: "9814b859e4b6f32188fc3e89f77ea124"
server_key_iv: "af80bf0da6ee29896853573936c34f57"
server_key_version: "kuduclusterkey@0"
I20260811 18:42:49.924206 21024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:42:49.944331 21024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:42:49.949383 21024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:42:49.949748 21024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:42:49.950938 21024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:42:49.952911 21024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:42:49.953022 21024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:42:49.953213 21024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:42:49.953300 21024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 18:42:50.034701 21024 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:35729
I20260811 18:42:50.034745 21296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:35729 every 8 connection(s)
I20260811 18:42:50.036443 21024 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:42:50.041742 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 21024
I20260811 18:42:50.041991 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:42:50.042709 31462 external_mini_cluster.cc:1428] Setting key b23e9273ce9cd90ba2d614a3dd548b0e
Aug 11 18:42:50 dist-test-slave-z8x9 krb5kdc[19637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473769, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.185.131@KRBTEST.COM; defaulting to no policy
I20260811 18:42:50.089037 21302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:42:50.044266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:57633 (local address 127.30.185.190:39639)
0811 18:42:50.058469 (+ 14203us) server_negotiation.cc:206] Beginning negotiation
0811 18:42:50.058477 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:42:50.058505 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:42:50.058581 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:42:50.058588 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:42:50.058686 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:42:50.058828 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:42:50.064286 (+ 5458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:50.065255 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:50.073335 (+ 8080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:50.074048 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:50.074204 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:42:50.079683 (+ 5479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:42:50.079691 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:42:50.079697 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:42:50.079747 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:42:50.083971 (+ 4224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:42:50.085428 (+ 1457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:42:50.085435 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:42:50.085439 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:42:50.085675 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:42:50.086285 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:42:50.086291 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:42:50.086294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:42:50.086892 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:42:50.087037 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 18:42:50.088119 (+ 1082us) server_negotiation.cc:299] Negotiation successful
0811 18:42:50.088380 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14035,"thread_start_us":14659,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.131@KRBTEST.COM" created.
I20260811 18:42:50.090018 21305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:42:50.045749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39639 (local address 127.30.185.130:57633)
0811 18:42:50.051674 (+ 5925us) negotiation.cc:107] Waiting for socket to connect
0811 18:42:50.051737 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 18:42:50.054164 (+ 2427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:42:50.061654 (+ 7490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:42:50.061672 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:42:50.062759 (+ 1087us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:42:50.064046 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:42:50.064072 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:42:50.065414 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:42:50.065419 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:42:50.067577 (+ 2158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:42:50.067594 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:42:50.074579 (+ 6985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:42:50.074588 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:42:50.074934 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:42:50.075798 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:42:50.075831 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:42:50.079349 (+ 3518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:42:50.084288 (+ 4939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:42:50.084299 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:42:50.084321 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:42:50.085264 (+ 943us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:42:50.085857 (+ 593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:42:50.085863 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:42:50.085867 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:42:50.086134 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:42:50.087064 (+ 930us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:42:50.087073 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:42:50.087602 (+ 529us) client_negotiation.cc:769] Sending connection context
0811 18:42:50.087935 (+ 333us) client_negotiation.cc:240] Negotiation successful
0811 18:42:50.088540 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5554,"thread_start_us":5311,"threads_started":1}
I20260811 18:42:50.098670 21297 heartbeater.cc:347] Connected to a master server at 127.30.185.190:39639
I20260811 18:42:50.099051 21297 heartbeater.cc:464] Registering TS with master...
I20260811 18:42:50.099978 21297 heartbeater.cc:511] Master 127.30.185.190:39639 requested a full tablet report, sending...
I20260811 18:42:50.101918 20559 ts_manager.cc:194] Registered new tserver with Master: f50641a15d544c4e93b99496c5e91d2c (127.30.185.130:35729)
I20260811 18:42:50.102969 20559 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:57633
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
I20260811 18:42:50.137854 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.185.148:47575/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:39639
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:33305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:42:50.228684 21011 heartbeater.cc:503] Master 127.30.185.190:39639 was elected leader, sending a full tablet report...
W20260811 18:42:50.643430 21313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:42:50.643836 21313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:42:50.644076 21313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:42:50.652128 21313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:42:50.652271 21313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:42:50.652451 21313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:42:50.666484 21313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:33305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.185.148:47575/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:39639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:42:50.668572 21313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:42:50.670679 21313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:42:50.688166 21328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:42:50.688429 21330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:42:50.688452 21333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:42:50.690083 21313 server_base.cc:1034] running on GCE node
I20260811 18:42:50.690761 21313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:42:50.691794 21313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:42:50.693032 21313 hybrid_clock.cc:648] HybridClock initialized: now 1786473770692977 us; error 56 us; skew 500 ppm
I20260811 18:42:50.693424 21313 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:42:50 dist-test-slave-z8x9 krb5kdc[19637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473770, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:42:50.696920 21313 init.cc:377] Logged in from keytab as kudu/127.30.185.131@KRBTEST.COM (short username kudu)
I20260811 18:42:50.699043 21313 webserver.cc:460] Webserver started at http://127.30.185.131:35605/ using document root <none> and password file <none>
I20260811 18:42:50.699801 21313 fs_manager.cc:359] Metadata directory not provided
I20260811 18:42:50.699951 21313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:42:50.700371 21313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:42:50 dist-test-slave-z8x9 krb5kdc[19637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473770, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
I20260811 18:42:50.718472 21313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "d9a1f6875aec446685b9be026e2b168a"
format_stamp: "Formatted at 2026-08-11 18:42:50 on dist-test-slave-z8x9"
server_key: "3ff5ebdec258b53b82481014075ef8f9"
server_key_iv: "86f86979f93723ec330c9fe8f6fab2f2"
server_key_version: "kuduclusterkey@0"
I20260811 18:42:50.719786 21313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "d9a1f6875aec446685b9be026e2b168a"
format_stamp: "Formatted at 2026-08-11 18:42:50 on dist-test-slave-z8x9"
server_key: "3ff5ebdec258b53b82481014075ef8f9"
server_key_iv: "86f86979f93723ec330c9fe8f6fab2f2"
server_key_version: "kuduclusterkey@0"
I20260811 18:42:50.727365 21313 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260811 18:42:50.743023 21345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:42:50.745158 21313 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:42:50.745414 21313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "d9a1f6875aec446685b9be026e2b168a"
format_stamp: "Formatted at 2026-08-11 18:42:50 on dist-test-slave-z8x9"
server_key: "3ff5ebdec258b53b82481014075ef8f9"
server_key_iv: "86f86979f93723ec330c9fe8f6fab2f2"
server_key_version: "kuduclusterkey@0"
I20260811 18:42:50.745697 21313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:42:50.772713 21313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:42:50.777473 21313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:42:50.777765 21313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:42:50.778820 21313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:42:50.780469 21313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:42:50.780568 21313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:42:50.780692 21313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:42:50.780742 21313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:42:50.828627 21313 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:40349
I20260811 18:42:50.828723 21559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:40349 every 8 connection(s)
I20260811 18:42:50.830189 21313 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:42:50.836125 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 21313
I20260811 18:42:50.836417 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:42:50.837294 31462 external_mini_cluster.cc:1428] Setting key 15dfc1f4e8729f11a8623a3e2d74d2d3
Aug 11 18:42:50 dist-test-slave-z8x9 krb5kdc[19637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473770, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:42:50.853845 21563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:42:50.833242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:37473 (local address 127.30.185.190:39639)
0811 18:42:50.833979 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0811 18:42:50.833984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:42:50.835091 (+ 1107us) server_negotiation.cc:407] Connection header received
0811 18:42:50.838304 (+ 3213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:42:50.838311 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:42:50.838402 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:42:50.838528 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:42:50.840430 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:50.841290 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:50.842911 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:50.843487 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:50.843624 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:42:50.847345 (+ 3721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:42:50.847352 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:42:50.847356 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:42:50.847389 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:42:50.850108 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:42:50.850839 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:42:50.850861 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:42:50.850864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:42:50.850998 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:42:50.851570 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:42:50.851579 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:42:50.851585 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:42:50.851913 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:42:50.852252 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0811 18:42:50.853089 (+ 837us) server_negotiation.cc:299] Negotiation successful
0811 18:42:50.853420 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":395,"threads_started":1}
I20260811 18:42:50.855132 21564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:42:50.833745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:39639 (local address 127.30.185.131:37473)
0811 18:42:50.834928 (+ 1183us) negotiation.cc:107] Waiting for socket to connect
0811 18:42:50.834976 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 18:42:50.837953 (+ 2977us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:42:50.838860 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:42:50.838873 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:42:50.839553 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:42:50.840250 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:42:50.840270 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:42:50.841460 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:42:50.841465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:42:50.842723 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:42:50.842744 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:42:50.843798 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:42:50.843810 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:42:50.844172 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:42:50.844844 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:42:50.844874 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:42:50.847115 (+ 2241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:42:50.850246 (+ 3131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:42:50.850255 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:42:50.850272 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:42:50.850710 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:42:50.851134 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:42:50.851139 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:42:50.851142 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:42:50.851468 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:42:50.852162 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:42:50.852171 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:42:50.852562 (+ 391us) client_negotiation.cc:769] Sending connection context
0811 18:42:50.853007 (+ 445us) client_negotiation.cc:240] Negotiation successful
0811 18:42:50.853721 (+ 714us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"thread_start_us":637,"threads_started":1}
I20260811 18:42:50.856531 21560 heartbeater.cc:347] Connected to a master server at 127.30.185.190:39639
I20260811 18:42:50.856850 21560 heartbeater.cc:464] Registering TS with master...
I20260811 18:42:50.858106 21560 heartbeater.cc:511] Master 127.30.185.190:39639 requested a full tablet report, sending...
I20260811 18:42:50.859730 20559 ts_manager.cc:194] Registered new tserver with Master: d9a1f6875aec446685b9be026e2b168a (127.30.185.131:40349)
I20260811 18:42:50.860561 20559 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:37473
I20260811 18:42:50.865487 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:42:50.885049 21563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:42:50.870702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49506 (local address 127.30.185.190:39639)
0811 18:42:50.870918 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 18:42:50.870926 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:42:50.871568 (+ 642us) server_negotiation.cc:407] Connection header received
0811 18:42:50.871703 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:42:50.871706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:42:50.871796 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:42:50.871879 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:42:50.873686 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:50.874431 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:50.875856 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:50.876276 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:50.876379 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:42:50.879218 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:42:50.879223 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:42:50.879226 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:42:50.879255 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:42:50.882298 (+ 3043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:42:50.883128 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:42:50.883131 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:42:50.883133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:42:50.883251 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:42:50.883750 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:42:50.883755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:42:50.883757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:42:50.884068 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:42:50.884304 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 18:42:50.884610 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 18:42:50.884805 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 18:42:50.889542 20559 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49506:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:42:50.898324 20559 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:42:50.927949 21586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:42:50.912729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59582 (local address 127.30.185.130:35729)
0811 18:42:50.915337 (+ 2608us) server_negotiation.cc:206] Beginning negotiation
0811 18:42:50.915350 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:42:50.915388 (+ 38us) server_negotiation.cc:407] Connection header received
0811 18:42:50.915509 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:42:50.915519 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:42:50.915857 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:42:50.916095 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:42:50.917919 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:50.920238 (+ 2319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:50.924515 (+ 4277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:50.926235 (+ 1720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:50.926414 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:42:50.926630 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 18:42:50.927258 (+ 628us) server_negotiation.cc:299] Negotiation successful
0811 18:42:50.927425 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2384,"thread_start_us":2237,"threads_started":1}
I20260811 18:42:50.928072 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:42:50.912481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:35729 (local address 127.0.0.1:59582)
0811 18:42:50.914544 (+ 2063us) negotiation.cc:107] Waiting for socket to connect
0811 18:42:50.914619 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 18:42:50.914883 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:42:50.916463 (+ 1580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:42:50.916475 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:42:50.916519 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:42:50.917510 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:42:50.917536 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:42:50.920465 (+ 2929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:42:50.920474 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:42:50.924206 (+ 3732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:42:50.924232 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:42:50.926772 (+ 2540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:42:50.926777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:42:50.926976 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:42:50.927015 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 18:42:50.927192 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 18:42:50.927393 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1749,"mutex_wait_us":21,"thread_start_us":1556,"threads_started":1}
I20260811 18:42:50.933089 21179 tablet_service.cc:1467] Processing CreateTablet for tablet 2bc25f374bba4b4b9929f9f80b52f5a4 (DEFAULT_TABLE table=test-table [id=f339fbe4174947ae81f3a9f707d60f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:42:50.934329 21591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:42:50.913596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46738 (local address 127.30.185.131:40349)
0811 18:42:50.923118 (+ 9522us) server_negotiation.cc:206] Beginning negotiation
0811 18:42:50.923128 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:42:50.925485 (+ 2357us) server_negotiation.cc:407] Connection header received
0811 18:42:50.925630 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:42:50.925643 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:42:50.926016 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:42:50.926267 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:42:50.927384 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:50.929584 (+ 2200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:50.931787 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:50.933062 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:50.933350 (+ 288us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:42:50.933536 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 18:42:50.933766 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 18:42:50.933934 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8876,"thread_start_us":8694,"threads_started":1}
I20260811 18:42:50.934336 21592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:42:50.913041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:40349 (local address 127.0.0.1:46738)
0811 18:42:50.925031 (+ 11990us) negotiation.cc:107] Waiting for socket to connect
0811 18:42:50.925059 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 18:42:50.925267 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:42:50.926393 (+ 1126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:42:50.926399 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:42:50.926426 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:42:50.927092 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:42:50.927107 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:42:50.929757 (+ 2650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:42:50.929763 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:42:50.931629 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:42:50.931643 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:42:50.933376 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:42:50.933384 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:42:50.933553 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:42:50.933598 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 18:42:50.933718 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 18:42:50.933866 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11786,"mutex_wait_us":29,"thread_start_us":13088,"threads_started":1}
I20260811 18:42:50.936223 21179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bc25f374bba4b4b9929f9f80b52f5a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:42:50.937595 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:42:50.915302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:40227 (local address 127.0.0.1:37554)
0811 18:42:50.926398 (+ 11096us) negotiation.cc:107] Waiting for socket to connect
0811 18:42:50.926422 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 18:42:50.926540 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:42:50.927712 (+ 1172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:42:50.927717 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:42:50.927754 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:42:50.928602 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:42:50.928618 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:42:50.930696 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:42:50.930702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:42:50.934513 (+ 3811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:42:50.934528 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:42:50.936563 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:42:50.936568 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:42:50.936684 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:42:50.936706 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 18:42:50.936821 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 18:42:50.936957 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10962,"mutex_wait_us":24,"thread_start_us":9356,"threads_started":1}
I20260811 18:42:50.937660 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:42:50.916202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37554 (local address 127.30.185.129:40227)
0811 18:42:50.923231 (+ 7029us) server_negotiation.cc:206] Beginning negotiation
0811 18:42:50.923245 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:42:50.926682 (+ 3437us) server_negotiation.cc:407] Connection header received
0811 18:42:50.926800 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:42:50.926809 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:42:50.927083 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:42:50.927287 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:42:50.928852 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:50.930542 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:50.934727 (+ 4185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:50.936046 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:50.936258 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:42:50.936439 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 18:42:50.936870 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 18:42:50.937049 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6753,"thread_start_us":6569,"threads_started":1}
I20260811 18:42:50.939287 21435 tablet_service.cc:1467] Processing CreateTablet for tablet 2bc25f374bba4b4b9929f9f80b52f5a4 (DEFAULT_TABLE table=test-table [id=f339fbe4174947ae81f3a9f707d60f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:42:50.941392 21435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bc25f374bba4b4b9929f9f80b52f5a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:42:50.941339 20865 tablet_service.cc:1467] Processing CreateTablet for tablet 2bc25f374bba4b4b9929f9f80b52f5a4 (DEFAULT_TABLE table=test-table [id=f339fbe4174947ae81f3a9f707d60f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:42:50.942657 20865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bc25f374bba4b4b9929f9f80b52f5a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:42:50.957957 21597 tablet_bootstrap.cc:492] T 2bc25f374bba4b4b9929f9f80b52f5a4 P 6737218bdfec48958ba94ab64a85da51: Bootstrap starting.
I20260811 18:42:50.959715 21595 tablet_bootstrap.cc:492] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c: Bootstrap starting.
I20260811 18:42:50.961951 21598 tablet_bootstrap.cc:492] T 2bc25f374bba4b4b9929f9f80b52f5a4 P d9a1f6875aec446685b9be026e2b168a: Bootstrap starting.
I20260811 18:42:50.963702 21597 tablet_bootstrap.cc:654] T 2bc25f374bba4b4b9929f9f80b52f5a4 P 6737218bdfec48958ba94ab64a85da51: Neither blocks nor log segments found. Creating new log.
I20260811 18:42:50.964002 21595 tablet_bootstrap.cc:654] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c: Neither blocks nor log segments found. Creating new log.
I20260811 18:42:50.965013 21595 log.cc:824] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:42:50.965122 21597 log.cc:824] T 2bc25f374bba4b4b9929f9f80b52f5a4 P 6737218bdfec48958ba94ab64a85da51: Log is configured to *not* fsync() on all Append() calls
I20260811 18:42:50.965936 21598 tablet_bootstrap.cc:654] T 2bc25f374bba4b4b9929f9f80b52f5a4 P d9a1f6875aec446685b9be026e2b168a: Neither blocks nor log segments found. Creating new log.
I20260811 18:42:50.967224 21598 log.cc:824] T 2bc25f374bba4b4b9929f9f80b52f5a4 P d9a1f6875aec446685b9be026e2b168a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:42:50.968626 21595 tablet_bootstrap.cc:492] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c: No bootstrap required, opened a new log
I20260811 18:42:50.968688 21597 tablet_bootstrap.cc:492] T 2bc25f374bba4b4b9929f9f80b52f5a4 P 6737218bdfec48958ba94ab64a85da51: No bootstrap required, opened a new log
I20260811 18:42:50.968921 21595 ts_tablet_manager.cc:1397] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 18:42:50.969022 21597 ts_tablet_manager.cc:1397] T 2bc25f374bba4b4b9929f9f80b52f5a4 P 6737218bdfec48958ba94ab64a85da51: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260811 18:42:50.970736 21598 tablet_bootstrap.cc:492] T 2bc25f374bba4b4b9929f9f80b52f5a4 P d9a1f6875aec446685b9be026e2b168a: No bootstrap required, opened a new log
I20260811 18:42:50.971014 21598 ts_tablet_manager.cc:1397] T 2bc25f374bba4b4b9929f9f80b52f5a4 P d9a1f6875aec446685b9be026e2b168a: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 18:42:50.977149 21595 raft_consensus.cc:348] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50641a15d544c4e93b99496c5e91d2c" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 35729 } } peers { permanent_uuid: "d9a1f6875aec446685b9be026e2b168a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40349 } } peers { permanent_uuid: "6737218bdfec48958ba94ab64a85da51" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 40227 } }
I20260811 18:42:50.977648 21595 raft_consensus.cc:374] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:42:50.977773 21595 raft_consensus.cc:729] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f50641a15d544c4e93b99496c5e91d2c, State: Initialized, Role: FOLLOWER
I20260811 18:42:50.978422 21595 consensus_queue.cc:260] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50641a15d544c4e93b99496c5e91d2c" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 35729 } } peers { permanent_uuid: "d9a1f6875aec446685b9be026e2b168a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40349 } } peers { permanent_uuid: "6737218bdfec48958ba94ab64a85da51" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 40227 } }
I20260811 18:42:50.979781 21297 heartbeater.cc:503] Master 127.30.185.190:39639 was elected leader, sending a full tablet report...
I20260811 18:42:50.980336 21595 ts_tablet_manager.cc:1428] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c: Time spent starting tablet: real 0.011s user 0.005s sys 0.007s
I20260811 18:42:50.980942 21597 raft_consensus.cc:348] T 2bc25f374bba4b4b9929f9f80b52f5a4 P 6737218bdfec48958ba94ab64a85da51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50641a15d544c4e93b99496c5e91d2c" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 35729 } } peers { permanent_uuid: "d9a1f6875aec446685b9be026e2b168a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40349 } } peers { permanent_uuid: "6737218bdfec48958ba94ab64a85da51" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 40227 } }
I20260811 18:42:50.981674 21597 raft_consensus.cc:374] T 2bc25f374bba4b4b9929f9f80b52f5a4 P 6737218bdfec48958ba94ab64a85da51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:42:50.981880 21597 raft_consensus.cc:729] T 2bc25f374bba4b4b9929f9f80b52f5a4 P 6737218bdfec48958ba94ab64a85da51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6737218bdfec48958ba94ab64a85da51, State: Initialized, Role: FOLLOWER
I20260811 18:42:50.982426 21598 raft_consensus.cc:348] T 2bc25f374bba4b4b9929f9f80b52f5a4 P d9a1f6875aec446685b9be026e2b168a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50641a15d544c4e93b99496c5e91d2c" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 35729 } } peers { permanent_uuid: "d9a1f6875aec446685b9be026e2b168a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40349 } } peers { permanent_uuid: "6737218bdfec48958ba94ab64a85da51" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 40227 } }
I20260811 18:42:50.982667 21597 consensus_queue.cc:260] T 2bc25f374bba4b4b9929f9f80b52f5a4 P 6737218bdfec48958ba94ab64a85da51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50641a15d544c4e93b99496c5e91d2c" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 35729 } } peers { permanent_uuid: "d9a1f6875aec446685b9be026e2b168a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40349 } } peers { permanent_uuid: "6737218bdfec48958ba94ab64a85da51" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 40227 } }
I20260811 18:42:50.983109 21598 raft_consensus.cc:374] T 2bc25f374bba4b4b9929f9f80b52f5a4 P d9a1f6875aec446685b9be026e2b168a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:42:50.983307 21598 raft_consensus.cc:729] T 2bc25f374bba4b4b9929f9f80b52f5a4 P d9a1f6875aec446685b9be026e2b168a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9a1f6875aec446685b9be026e2b168a, State: Initialized, Role: FOLLOWER
I20260811 18:42:50.984135 21598 consensus_queue.cc:260] T 2bc25f374bba4b4b9929f9f80b52f5a4 P d9a1f6875aec446685b9be026e2b168a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50641a15d544c4e93b99496c5e91d2c" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 35729 } } peers { permanent_uuid: "d9a1f6875aec446685b9be026e2b168a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40349 } } peers { permanent_uuid: "6737218bdfec48958ba94ab64a85da51" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 40227 } }
I20260811 18:42:50.986124 21597 ts_tablet_manager.cc:1428] T 2bc25f374bba4b4b9929f9f80b52f5a4 P 6737218bdfec48958ba94ab64a85da51: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
I20260811 18:42:50.988749 21560 heartbeater.cc:503] Master 127.30.185.190:39639 was elected leader, sending a full tablet report...
I20260811 18:42:50.989509 21598 ts_tablet_manager.cc:1428] T 2bc25f374bba4b4b9929f9f80b52f5a4 P d9a1f6875aec446685b9be026e2b168a: Time spent starting tablet: real 0.018s user 0.015s sys 0.005s
I20260811 18:42:50.994282 21606 raft_consensus.cc:482] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:42:50.994560 21606 raft_consensus.cc:504] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50641a15d544c4e93b99496c5e91d2c" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 35729 } } peers { permanent_uuid: "d9a1f6875aec446685b9be026e2b168a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40349 } } peers { permanent_uuid: "6737218bdfec48958ba94ab64a85da51" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 40227 } }
I20260811 18:42:50.996263 21606 leader_election.cc:288] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9a1f6875aec446685b9be026e2b168a (127.30.185.131:40349), 6737218bdfec48958ba94ab64a85da51 (127.30.185.129:40227)
I20260811 18:42:51.004823 21591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:42:50.996792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:57627 (local address 127.30.185.131:40349)
0811 18:42:50.997017 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 18:42:50.997025 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:42:50.997736 (+ 711us) server_negotiation.cc:407] Connection header received
0811 18:42:50.997962 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:42:50.997967 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:42:50.998026 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:42:50.998121 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:42:50.998948 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:51.000443 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:51.002523 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:51.003767 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:51.003953 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:42:51.004057 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 18:42:51.004407 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 18:42:51.004532 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 18:42:51.005075 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:42:50.996432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:40349 (local address 127.30.185.130:57627)
0811 18:42:50.997620 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0811 18:42:50.997642 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:42:50.997809 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:42:50.998140 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:42:50.998144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:42:50.998173 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:42:50.998766 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:42:50.998783 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:42:51.000591 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:42:51.000594 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:42:51.002333 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:42:51.002346 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:42:51.004049 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:42:51.004055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:42:51.004245 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:42:51.004270 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:42:51.004380 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 18:42:51.004510 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1040,"mutex_wait_us":21,"thread_start_us":2778,"threads_started":1}
I20260811 18:42:51.005937 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:42:50.997053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:34141 (local address 127.30.185.129:40227)
0811 18:42:50.997289 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 18:42:50.997294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:42:50.999561 (+ 2267us) server_negotiation.cc:407] Connection header received
0811 18:42:50.999708 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:42:50.999713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:42:50.999790 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:42:50.999894 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:42:51.000363 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:51.001283 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:51.003748 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:51.004877 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:51.005090 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:42:51.005192 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 18:42:51.005656 (+ 464us) server_negotiation.cc:299] Negotiation successful
0811 18:42:51.005756 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:42:51.006182 21610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:42:50.996632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:40227 (local address 127.30.185.130:34141)
0811 18:42:50.999456 (+ 2824us) negotiation.cc:107] Waiting for socket to connect
0811 18:42:50.999471 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:42:50.999570 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:42:50.999913 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:42:50.999916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:42:50.999933 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:42:51.000205 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:42:51.000211 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:42:51.001452 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:42:51.001456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:42:51.003493 (+ 2037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:42:51.003509 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:42:51.005304 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:42:51.005309 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:42:51.005443 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:42:51.005495 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 18:42:51.005637 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 18:42:51.005798 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2722,"mutex_wait_us":2,"thread_start_us":513,"threads_started":1}
I20260811 18:42:51.006680 21478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bc25f374bba4b4b9929f9f80b52f5a4" candidate_uuid: "f50641a15d544c4e93b99496c5e91d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9a1f6875aec446685b9be026e2b168a" is_pre_election: true
I20260811 18:42:51.007133 20912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bc25f374bba4b4b9929f9f80b52f5a4" candidate_uuid: "f50641a15d544c4e93b99496c5e91d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6737218bdfec48958ba94ab64a85da51" is_pre_election: true
I20260811 18:42:51.007519 21478 raft_consensus.cc:2454] T 2bc25f374bba4b4b9929f9f80b52f5a4 P d9a1f6875aec446685b9be026e2b168a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f50641a15d544c4e93b99496c5e91d2c in term 0.
I20260811 18:42:51.007710 20912 raft_consensus.cc:2454] T 2bc25f374bba4b4b9929f9f80b52f5a4 P 6737218bdfec48958ba94ab64a85da51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f50641a15d544c4e93b99496c5e91d2c in term 0.
I20260811 18:42:51.008263 21086 leader_election.cc:302] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d9a1f6875aec446685b9be026e2b168a, f50641a15d544c4e93b99496c5e91d2c; no voters:
I20260811 18:42:51.008682 21606 raft_consensus.cc:2781] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:42:51.008844 21606 raft_consensus.cc:482] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:42:51.009037 21606 raft_consensus.cc:3037] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:42:51.011446 21606 raft_consensus.cc:504] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50641a15d544c4e93b99496c5e91d2c" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 35729 } } peers { permanent_uuid: "d9a1f6875aec446685b9be026e2b168a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40349 } } peers { permanent_uuid: "6737218bdfec48958ba94ab64a85da51" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 40227 } }
I20260811 18:42:51.012126 21606 leader_election.cc:288] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c [CANDIDATE]: Term 1 election: Requested vote from peers d9a1f6875aec446685b9be026e2b168a (127.30.185.131:40349), 6737218bdfec48958ba94ab64a85da51 (127.30.185.129:40227)
I20260811 18:42:51.012578 21478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bc25f374bba4b4b9929f9f80b52f5a4" candidate_uuid: "f50641a15d544c4e93b99496c5e91d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9a1f6875aec446685b9be026e2b168a"
I20260811 18:42:51.012810 20912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bc25f374bba4b4b9929f9f80b52f5a4" candidate_uuid: "f50641a15d544c4e93b99496c5e91d2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6737218bdfec48958ba94ab64a85da51"
I20260811 18:42:51.012908 21478 raft_consensus.cc:3037] T 2bc25f374bba4b4b9929f9f80b52f5a4 P d9a1f6875aec446685b9be026e2b168a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:42:51.013135 20912 raft_consensus.cc:3037] T 2bc25f374bba4b4b9929f9f80b52f5a4 P 6737218bdfec48958ba94ab64a85da51 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:42:51.016696 20912 raft_consensus.cc:2454] T 2bc25f374bba4b4b9929f9f80b52f5a4 P 6737218bdfec48958ba94ab64a85da51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f50641a15d544c4e93b99496c5e91d2c in term 1.
I20260811 18:42:51.016696 21478 raft_consensus.cc:2454] T 2bc25f374bba4b4b9929f9f80b52f5a4 P d9a1f6875aec446685b9be026e2b168a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f50641a15d544c4e93b99496c5e91d2c in term 1.
I20260811 18:42:51.017259 21082 leader_election.cc:302] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6737218bdfec48958ba94ab64a85da51, d9a1f6875aec446685b9be026e2b168a, f50641a15d544c4e93b99496c5e91d2c; no voters:
I20260811 18:42:51.017649 21606 raft_consensus.cc:2781] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:42:51.018167 21606 raft_consensus.cc:686] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c [term 1 LEADER]: Becoming Leader. State: Replica: f50641a15d544c4e93b99496c5e91d2c, State: Running, Role: LEADER
I20260811 18:42:51.018814 21606 consensus_queue.cc:237] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50641a15d544c4e93b99496c5e91d2c" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 35729 } } peers { permanent_uuid: "d9a1f6875aec446685b9be026e2b168a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40349 } } peers { permanent_uuid: "6737218bdfec48958ba94ab64a85da51" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 40227 } }
I20260811 18:42:51.024247 20558 catalog_manager.cc:5495] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c reported cstate change: term changed from 0 to 1, leader changed from <none> to f50641a15d544c4e93b99496c5e91d2c (127.30.185.130). New cstate: current_term: 1 leader_uuid: "f50641a15d544c4e93b99496c5e91d2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50641a15d544c4e93b99496c5e91d2c" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 35729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9a1f6875aec446685b9be026e2b168a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6737218bdfec48958ba94ab64a85da51" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 40227 } health_report { overall_health: UNKNOWN } } }
W20260811 18:42:51.045680 21300 tablet.cc:2369] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:42:51.060771 21586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:42:51.055141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59584 (local address 127.30.185.130:35729)
0811 18:42:51.055345 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 18:42:51.055350 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:42:51.055370 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:42:51.055454 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:42:51.055459 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:42:51.055548 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:42:51.055686 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:42:51.056043 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:51.056874 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:51.058391 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:51.058819 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:51.058959 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:42:51.059570 (+ 611us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:42:51.060074 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:42:51.060232 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 18:42:51.060336 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 18:42:51.060429 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 18:42:51.069963 21478 raft_consensus.cc:1261] T 2bc25f374bba4b4b9929f9f80b52f5a4 P d9a1f6875aec446685b9be026e2b168a [term 1 FOLLOWER]: Refusing update from remote peer f50641a15d544c4e93b99496c5e91d2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:42:51.069963 20912 raft_consensus.cc:1261] T 2bc25f374bba4b4b9929f9f80b52f5a4 P 6737218bdfec48958ba94ab64a85da51 [term 1 FOLLOWER]: Refusing update from remote peer f50641a15d544c4e93b99496c5e91d2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:42:51.071080 21613 consensus_queue.cc:1035] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6737218bdfec48958ba94ab64a85da51" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 40227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:42:51.071475 21606 consensus_queue.cc:1035] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9a1f6875aec446685b9be026e2b168a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 40349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:42:51.082029 21562 tablet.cc:2369] T 2bc25f374bba4b4b9929f9f80b52f5a4 P d9a1f6875aec446685b9be026e2b168a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:42:51.085875 21616 mvcc.cc:205] Tried to move back new op lower bound from 7317396566291075072 to 7317396566098190336. Current Snapshot: MvccSnapshot[applied={T|T < 7317396566291075072}]
I20260811 18:42:51.101029 20556 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:42:51.101414 20556 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:42:51.105810 20556 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1aa9bed12924aab8fa1615d9ab3923b: Sending DeleteTablet for 3 replicas of tablet 2bc25f374bba4b4b9929f9f80b52f5a4
I20260811 18:42:51.107154 21179 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bc25f374bba4b4b9929f9f80b52f5a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:42:51 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:59582
I20260811 18:42:51.107450 21435 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bc25f374bba4b4b9929f9f80b52f5a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:42:51 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:46738
I20260811 18:42:51.108036 20865 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bc25f374bba4b4b9929f9f80b52f5a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:42:51 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:37554
I20260811 18:42:51.110098 21628 tablet_replica.cc:330] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c: stopping tablet replica
I20260811 18:42:51.110333 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 20712
I20260811 18:42:51.110713 21628 raft_consensus.cc:2229] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:42:51.112159 21628 raft_consensus.cc:2258] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:42:51.115578 21628 ts_tablet_manager.cc:1905] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:42:51.119072 21633 tablet_replica.cc:330] T 2bc25f374bba4b4b9929f9f80b52f5a4 P 6737218bdfec48958ba94ab64a85da51: stopping tablet replica
I20260811 18:42:51.119550 21633 raft_consensus.cc:2229] T 2bc25f374bba4b4b9929f9f80b52f5a4 P 6737218bdfec48958ba94ab64a85da51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:42:51.120122 21633 raft_consensus.cc:2258] T 2bc25f374bba4b4b9929f9f80b52f5a4 P 6737218bdfec48958ba94ab64a85da51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:42:51.121122 21629 tablet_replica.cc:330] T 2bc25f374bba4b4b9929f9f80b52f5a4 P d9a1f6875aec446685b9be026e2b168a: stopping tablet replica
I20260811 18:42:51.121598 21629 raft_consensus.cc:2229] T 2bc25f374bba4b4b9929f9f80b52f5a4 P d9a1f6875aec446685b9be026e2b168a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:42:51.122100 21629 raft_consensus.cc:2258] T 2bc25f374bba4b4b9929f9f80b52f5a4 P d9a1f6875aec446685b9be026e2b168a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:42:51.124054 21633 ts_tablet_manager.cc:1905] T 2bc25f374bba4b4b9929f9f80b52f5a4 P 6737218bdfec48958ba94ab64a85da51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:42:51.124883 21628 ts_tablet_manager.cc:1918] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:42:51.125065 21628 log.cc:1196] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/2bc25f374bba4b4b9929f9f80b52f5a4
I20260811 18:42:51.125732 21628 ts_tablet_manager.cc:1939] T 2bc25f374bba4b4b9929f9f80b52f5a4 P f50641a15d544c4e93b99496c5e91d2c: Deleting consensus metadata
I20260811 18:42:51.127754 20528 catalog_manager.cc:4841] TS f50641a15d544c4e93b99496c5e91d2c (127.30.185.130:35729): tablet 2bc25f374bba4b4b9929f9f80b52f5a4 (table test-table [id=f339fbe4174947ae81f3a9f707d60f6e]) successfully deleted
I20260811 18:42:51.129352 21629 ts_tablet_manager.cc:1905] T 2bc25f374bba4b4b9929f9f80b52f5a4 P d9a1f6875aec446685b9be026e2b168a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:42:51.135493 21629 ts_tablet_manager.cc:1918] T 2bc25f374bba4b4b9929f9f80b52f5a4 P d9a1f6875aec446685b9be026e2b168a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:42:51.135812 21629 log.cc:1196] T 2bc25f374bba4b4b9929f9f80b52f5a4 P d9a1f6875aec446685b9be026e2b168a: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/2bc25f374bba4b4b9929f9f80b52f5a4
I20260811 18:42:51.136633 21629 ts_tablet_manager.cc:1939] T 2bc25f374bba4b4b9929f9f80b52f5a4 P d9a1f6875aec446685b9be026e2b168a: Deleting consensus metadata
I20260811 18:42:51.139130 20530 catalog_manager.cc:4841] TS d9a1f6875aec446685b9be026e2b168a (127.30.185.131:40349): tablet 2bc25f374bba4b4b9929f9f80b52f5a4 (table test-table [id=f339fbe4174947ae81f3a9f707d60f6e]) successfully deleted
I20260811 18:42:51.139789 21633 ts_tablet_manager.cc:1918] T 2bc25f374bba4b4b9929f9f80b52f5a4 P 6737218bdfec48958ba94ab64a85da51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:42:51.139999 21633 log.cc:1196] T 2bc25f374bba4b4b9929f9f80b52f5a4 P 6737218bdfec48958ba94ab64a85da51: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786473610573616-31462-0/minicluster-data/ts-0/wal/wals/2bc25f374bba4b4b9929f9f80b52f5a4
I20260811 18:42:51.140590 21633 ts_tablet_manager.cc:1939] T 2bc25f374bba4b4b9929f9f80b52f5a4 P 6737218bdfec48958ba94ab64a85da51: Deleting consensus metadata
I20260811 18:42:51.142616 20527 catalog_manager.cc:4841] TS 6737218bdfec48958ba94ab64a85da51 (127.30.185.129:40227): tablet 2bc25f374bba4b4b9929f9f80b52f5a4 (table test-table [id=f339fbe4174947ae81f3a9f707d60f6e]) successfully deleted
Aug 11 18:42:51 dist-test-slave-z8x9 krb5kdc[19637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.129@KRBTEST.COM
I20260811 18:42:51.155644 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:42:51.112144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37564 (local address 127.30.185.129:40227)
0811 18:42:51.112334 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 18:42:51.112340 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:42:51.136847 (+ 24507us) server_negotiation.cc:407] Connection header received
0811 18:42:51.137289 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:42:51.137296 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:42:51.137405 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:42:51.137492 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:42:51.140009 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:51.140889 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:51.142531 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:51.143224 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:51.143364 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:42:51.148050 (+ 4686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:42:51.148066 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:42:51.148088 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:42:51.148141 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:42:51.152387 (+ 4246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:42:51.153201 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:42:51.153212 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:42:51.153216 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:42:51.153391 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:42:51.153785 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:42:51.153791 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:42:51.153795 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:42:51.154300 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:42:51.154624 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 18:42:51.154946 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 18:42:51.155223 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:42:51.156560 20998 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:42:51.261611 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 20712
I20260811 18:42:51.291020 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 21024
Aug 11 18:42:51 dist-test-slave-z8x9 krb5kdc[19637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.130@KRBTEST.COM
I20260811 18:42:51.309798 21586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:42:51.291858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59594 (local address 127.30.185.130:35729)
0811 18:42:51.292036 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 18:42:51.292041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:42:51.292064 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:42:51.292315 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:42:51.292319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:42:51.292389 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:42:51.292471 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:42:51.294344 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:51.295360 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:51.297331 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:51.297879 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:51.297987 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:42:51.303223 (+ 5236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:42:51.303238 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:42:51.303253 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:42:51.303311 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:42:51.306505 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:42:51.307292 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:42:51.307300 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:42:51.307302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:42:51.307481 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:42:51.307949 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:42:51.307955 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:42:51.307959 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:42:51.308475 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:42:51.308886 (+ 411us) server_negotiation.cc:1035] Waiting for connection context
0811 18:42:51.309106 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 18:42:51.309385 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 18:42:51.310565 21288 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:42:51.425450 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 21024
I20260811 18:42:51.456559 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 21313
Aug 11 18:42:51 dist-test-slave-z8x9 krb5kdc[19637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.131@KRBTEST.COM
I20260811 18:42:51.471737 21591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:42:51.457346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46750 (local address 127.30.185.131:40349)
0811 18:42:51.457513 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 18:42:51.457518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:42:51.457541 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:42:51.457711 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:42:51.457715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:42:51.457781 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:42:51.457868 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:42:51.459188 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:51.459915 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:51.461126 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:42:51.461616 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:42:51.461707 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:42:51.466290 (+ 4583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:42:51.466306 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:42:51.466322 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:42:51.466370 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:42:51.469160 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:42:51.469878 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:42:51.469886 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:42:51.469888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:42:51.470015 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:42:51.470395 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:42:51.470398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:42:51.470400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:42:51.470800 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:42:51.471021 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 18:42:51.471273 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 18:42:51.471459 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:42:51.472404 21550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:42:51.561811 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 21313
I20260811 18:42:51.586921 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 20495
I20260811 18:42:51.588131 20603 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:42:51.704452 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 20495
I20260811 18:42:51.740366 31462 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 18:42:52.086864 31462 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 18:42:52.087170 31462 mini_ranger.cc:66] Stopping Ranger...
I20260811 18:42:52.443075 31462 mini_ranger.cc:68] Stopped Ranger
2026-08-11 18:42:52.443 UTC [19702] LOG: received smart shutdown request
2026-08-11 18:42:52.447 UTC [19702] LOG: background worker "logical replication launcher" (PID 19723) exited with exit code 1
2026-08-11 18:42:52.448 UTC [19707] LOG: shutting down
2026-08-11 18:42:52.622 UTC [19702] LOG: database system is shut down
2026-08-11T18:42:52Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53124 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:42:52 dist-test-slave-z8x9 krb5kdc[21657](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:42:52 dist-test-slave-z8x9 krb5kdc[21657](info): set up 2 sockets
Aug 11 18:42:52 dist-test-slave-z8x9 krb5kdc[21657](info): commencing operation
krb5kdc: starting...
W20260811 18:42:54.179690 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.362s user 0.005s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:42:54 dist-test-slave-z8x9 krb5kdc[21657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:42:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:42:54Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.185.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.185.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.185.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/rangeradmin_127.30.185.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.185.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/rangeradmin_127.30.185.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.185.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.185.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.185.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/rangerlookup_127.30.185.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.185.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/rangerlookup_127.30.185.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.185.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.185.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.185.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/HTTP_127.30.185.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.185.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/HTTP_127.30.185.148@KRBTEST.COM.keytab.
I20260811 18:42:54.600162 31462 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasky46AhN/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/postgres -l logfile start
2026-08-11 18:42:57.168 UTC [21706] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 18:42:57.168 UTC [21706] LOG: listening on IPv4 address "127.30.185.148", port 44669
2026-08-11 18:42:57.172 UTC [21706] LOG: listening on Unix socket "/tmp/.s.PGSQL.44669"
2026-08-11 18:42:57.205 UTC [21711] LOG: database system was shut down at 2026-08-11 18:42:55 UTC
2026-08-11 18:42:57.209 UTC [21706] LOG: database system is ready to accept connections
I20260811 18:42:58.514887 31462 mini_postgres.cc:96] Postgres bound to 44669
2026-08-11 18:42:58.539 UTC [21726] FATAL: database "slave" does not exist
127.30.185.148:44669 - accepting connections
I20260811 18:42:58.540258 31462 mini_ranger.cc:161] Starting Ranger...
I20260811 18:42:58.573935 31462 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 18:42:59.090180 31462 mini_ranger.cc:87] Created ranger Postgres database
I20260811 18:42:59.090313 31462 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ranger-admin
2026-08-11 18:42:59,399 [I] DB FLAVOR :POSTGRES
2026-08-11 18:42:59,400 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 18:42:59,400 [I] Checking connection..
2026-08-11 18:42:59,400 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:44669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 18:42:59,669 [I] Checking connection passed.
2026-08-11 18:42:59,669 [I] --------- Verifying version history table ---------
2026-08-11 18:42:59,669 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:44669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:42:59,944 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 18:42:59,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:44669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:43:00,220 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 18:43:00,220 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 18:43:00,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:44669/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 18:43:00.486 UTC [21860] WARNING: there is no transaction in progress
2026-08-11 18:43:00,496 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 18:43:00,497 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:44669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:43:00,751 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 18:43:00,752 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 18:43:00,752 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:44669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 18:43:01,023 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:44669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 18:43:01,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:44669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-z8x9.c.gcp-upstream.internal','N') ;"
2026-08-11 18:43:01,565 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:44669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 18:43:01,829 [I] Table x_portal_user does not exist in database ranger
2026-08-11 18:43:01,829 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:44669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 18:43:02,087 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 18:43:02,087 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:44669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 18:43:02,361 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:44669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 18:43:02,611 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 18:43:02,611 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:44669/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 18:43:03.658 UTC [22061] WARNING: there is no transaction in progress
2026-08-11 18:43:03.676 UTC [22061] WARNING: there is no transaction in progress
2026-08-11 18:43:03.694 UTC [22061] WARNING: there is no transaction in progress
2026-08-11 18:43:03.711 UTC [22061] WARNING: there is no transaction in progress
2026-08-11 18:43:03.727 UTC [22061] WARNING: there is no transaction in progress
2026-08-11 18:43:03.745 UTC [22061] WARNING: there is no transaction in progress
2026-08-11 18:43:03.807 UTC [22061] WARNING: there is no transaction in progress
2026-08-11 18:43:03.815 UTC [22061] WARNING: there is no transaction in progress
2026-08-11 18:43:03.831 UTC [22061] WARNING: there is no transaction in progress
2026-08-11 18:43:03.842 UTC [22061] WARNING: there is no transaction in progress
2026-08-11 18:43:03.854 UTC [22061] WARNING: there is no transaction in progress
2026-08-11 18:43:03.869 UTC [22061] WARNING: there is no transaction in progress
2026-08-11 18:43:03.879 UTC [22061] WARNING: there is no transaction in progress
2026-08-11 18:43:03.886 UTC [22061] WARNING: there is no transaction in progress
2026-08-11 18:43:03.893 UTC [22061] WARNING: there is no transaction in progress
2026-08-11 18:43:03.900 UTC [22061] WARNING: there is no transaction in progress
2026-08-11 18:43:04,616 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 18:43:04,616 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:44669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 18:43:04,873 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 18:43:04,874 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:44669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 18:43:05,163 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 18:43:05,164 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:44669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 18:43:05,445 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 18:43:05,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:44669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 18:43:05,709 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:44669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 18:43:05,991 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:44669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-z8x9.c.gcp-upstream.internal';"
2026-08-11 18:43:06,257 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 18:43:06,257 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:44669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 18:43:06,520 [I] DB_PATCHES have already been applied
I20260811 18:43:06.526777 31462 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ranger-admin:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasky46AhN/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 18:43:06.526870 31462 mini_ranger.cc:193] Using host: 127.30.185.148
I20260811 18:43:06.549146 31462 mini_ranger.cc:227] Ranger admin URL: http://127.30.185.148:44657
Aug 11, 2026 6:43:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:43:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:43:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 6:43:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 6:43:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 6:43:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 6:43:07 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 18:43:08.038868 31462 mini_ranger.cc:160] Time spent starting Ranger: real 9.499s user 0.001s sys 0.065s
Aug 11, 2026 6:43:28 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 6:43:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 6:43:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 6:43:28 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 6:43:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 6:43:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 6:43:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 6:43:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 6:43:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 6:43:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 6:43:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 6:43:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 6:43:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 6:43:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 6:43:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 6:43:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 6:43:28 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 6:43:29 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 18:43:30.244259 31462 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.185.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.185.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.185.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/rangerkms_127.30.185.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.185.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/rangerkms_127.30.185.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.185.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.185.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.185.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/HTTP_127.30.185.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.185.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/HTTP_127.30.185.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 18:43:30 dist-test-slave-z8x9 krb5kdc[21657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473810, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 18:43:30.465409 31462 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 18:43:30.493969 31462 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 18:43:31.375679 31462 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 18:43:31.375813 31462 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ranger-kms
I20260811 18:43:31.375876 31462 mini_ranger_kms.cc:227] Using postgres at 127.30.185.148:44669
2026-08-11 18:43:31,747 [I] DB FLAVOR :POSTGRES
2026-08-11 18:43:31,747 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 18:43:31,748 [I] Checking connection
2026-08-11 18:43:31,748 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:44669/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 18:43:32,024 [I] connection success
2026-08-11 18:43:32,024 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 18:43:32,024 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 18:43:32,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:44669/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 18:43:32,265 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 18:43:32,266 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 18:43:32,266 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 18:43:32,266 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.185.148:44669/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 18:43:32,546 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 18:43:33.049629 31462 mini_ranger_kms.cc:310] Created kms service
I20260811 18:43:33.285567 31462 mini_ranger_kms.cc:326] Created kudu user
I20260811 18:43:33.336572 31462 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 18:43:33 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473810, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
I20260811 18:43:33.805626 31462 mini_ranger_kms.cc:384] Added ranger policy
I20260811 18:43:33.806005 31462 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ranger-kms:/tmp/dist-test-tasky46AhN/build/asan/bin/postgresql.jar:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasky46AhN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 18:43:33.806064 31462 mini_ranger_kms.cc:240] Using host: 127.30.185.148
I20260811 18:43:33.833895 31462 mini_ranger_kms.cc:276] Ranger KMS PID: 22393
I20260811 18:43:33.834132 31462 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.185.148:41301
Aug 11, 2026 6:43:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:43:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:43:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 6:43:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 6:43:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 6:43:35 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 18:43:36 dist-test-slave-z8x9 krb5kdc[21657](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786473816, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.185.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 18:43:36 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473816, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.185.148@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
I20260811 18:43:36.799719 31462 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.334s user 0.057s sys 0.356s
I20260811 18:43:36.799988 31462 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 18:43:36.800184 31462 mini_ranger_kms.cc:401] 127.30.185.148:41301/kms/v1/keys
Aug 11 18:43:37 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473816, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.185.148@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
Aug 11 18:43:39 dist-test-slave-z8x9 krb5kdc[21657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:43:39.613951 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:41913
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:38877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:41913
--ranger_config_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.185.148:41301/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:43:39.961972 22479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:43:39.962348 22479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:43:39.962525 22479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:43:39.969854 22479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:43:39.970027 22479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:43:39.970144 22479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:43:39.970253 22479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:43:39.970359 22479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:43:39.980760 22479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.185.148:41301/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:41913
--ranger_config_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:41913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:43:39.982244 22479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:43:39.983614 22479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:43:39.995610 22493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:39.995715 22489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:39.995748 22487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:43:39.996738 22479 server_base.cc:1034] running on GCE node
I20260811 18:43:39.997671 22479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:43:39.999125 22479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:43:40.000358 22479 hybrid_clock.cc:648] HybridClock initialized: now 1786473820000311 us; error 47 us; skew 500 ppm
I20260811 18:43:40.000735 22479 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:43:39 dist-test-slave-z8x9 krb5kdc[21657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473819, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:43:40.004072 22479 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:43:40.005856 22479 webserver.cc:460] Webserver started at http://127.30.185.190:42199/ using document root <none> and password file <none>
I20260811 18:43:40.006486 22479 fs_manager.cc:359] Metadata directory not provided
I20260811 18:43:40.006587 22479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:43:40.006843 22479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:43:40 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473819, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
I20260811 18:43:40.066226 22479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "b04b3907879f447ab707149cb088f8b7"
format_stamp: "Formatted at 2026-08-11 18:43:40 on dist-test-slave-z8x9"
server_key: "94c9ed62be5aa551c8fe35b6e062c57f"
server_key_iv: "53701a5e9a149999753e628f5bcf024f"
server_key_version: "kuduclusterkey@0"
I20260811 18:43:40.067157 22479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "b04b3907879f447ab707149cb088f8b7"
format_stamp: "Formatted at 2026-08-11 18:43:40 on dist-test-slave-z8x9"
server_key: "94c9ed62be5aa551c8fe35b6e062c57f"
server_key_iv: "53701a5e9a149999753e628f5bcf024f"
server_key_version: "kuduclusterkey@0"
I20260811 18:43:40.072813 22479 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 18:43:40.093046 22508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:43:40.094908 22479 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 18:43:40.095152 22479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "b04b3907879f447ab707149cb088f8b7"
format_stamp: "Formatted at 2026-08-11 18:43:40 on dist-test-slave-z8x9"
server_key: "94c9ed62be5aa551c8fe35b6e062c57f"
server_key_iv: "53701a5e9a149999753e628f5bcf024f"
server_key_version: "kuduclusterkey@0"
I20260811 18:43:40.095439 22479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:43:40.132025 22479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:43:40.136934 22479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:43:40.137401 22479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:43:40.160247 22479 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:41913
I20260811 18:43:40.160249 22597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:41913 every 8 connection(s)
I20260811 18:43:40.161978 22479 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:43:40.166448 22598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:43:40.168533 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 22479
I20260811 18:43:40.168784 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:43:40.169638 31462 external_mini_cluster.cc:1428] Setting key bee3c74894708f7be2d41f9cca48ef55
I20260811 18:43:40.186318 22598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7: Bootstrap starting.
I20260811 18:43:40.190271 22598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7: Neither blocks nor log segments found. Creating new log.
I20260811 18:43:40.191426 22598 log.cc:824] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7: Log is configured to *not* fsync() on all Append() calls
I20260811 18:43:40.194711 22598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7: No bootstrap required, opened a new log
Aug 11 18:43:40 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:43:40.203827 22598 raft_consensus.cc:348] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b04b3907879f447ab707149cb088f8b7" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41913 } }
I20260811 18:43:40.204276 22598 raft_consensus.cc:374] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:43:40.204424 22598 raft_consensus.cc:729] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b04b3907879f447ab707149cb088f8b7, State: Initialized, Role: FOLLOWER
I20260811 18:43:40.205235 22598 consensus_queue.cc:260] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b04b3907879f447ab707149cb088f8b7" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41913 } }
I20260811 18:43:40.205578 22598 raft_consensus.cc:388] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:43:40.205781 22598 raft_consensus.cc:482] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:43:40.206014 22598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:43:40.206535 22607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:40.172010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47664 (local address 127.30.185.190:41913)
0811 18:43:40.186005 (+ 13995us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:40.186029 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:40.186454 (+ 425us) server_negotiation.cc:407] Connection header received
0811 18:43:40.188432 (+ 1978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:40.188459 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:40.188855 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:40.189331 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:40.190853 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:40.192265 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:40.193709 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:40.194373 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:40.194586 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:40.198666 (+ 4080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:40.198675 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:40.198689 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:40.198725 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:40.202209 (+ 3484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:40.202946 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:40.202953 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:40.202959 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:40.203110 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:40.203515 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:40.203520 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:40.203522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:40.204069 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:40.204401 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:40.204840 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 18:43:40.205299 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13697,"thread_start_us":13426,"threads_started":1}
I20260811 18:43:40.209218 22598 raft_consensus.cc:504] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b04b3907879f447ab707149cb088f8b7" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41913 } }
I20260811 18:43:40.209882 22598 leader_election.cc:302] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b04b3907879f447ab707149cb088f8b7; no voters:
I20260811 18:43:40.210520 22598 leader_election.cc:288] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:43:40.210667 22609 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:43:40.211433 22609 raft_consensus.cc:686] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [term 1 LEADER]: Becoming Leader. State: Replica: b04b3907879f447ab707149cb088f8b7, State: Running, Role: LEADER
I20260811 18:43:40.212021 22609 consensus_queue.cc:237] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b04b3907879f447ab707149cb088f8b7" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41913 } }
I20260811 18:43:40.212889 22598 sys_catalog.cc:564] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:43:40.215729 22615 sys_catalog.cc:455] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b04b3907879f447ab707149cb088f8b7. Latest consensus state: current_term: 1 leader_uuid: "b04b3907879f447ab707149cb088f8b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b04b3907879f447ab707149cb088f8b7" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41913 } } }
I20260811 18:43:40.216401 22615 sys_catalog.cc:458] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [sys.catalog]: This master's current role is: LEADER
I20260811 18:43:40.221369 22614 sys_catalog.cc:455] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b04b3907879f447ab707149cb088f8b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b04b3907879f447ab707149cb088f8b7" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41913 } } }
I20260811 18:43:40.221813 22614 sys_catalog.cc:458] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [sys.catalog]: This master's current role is: LEADER
I20260811 18:43:40.223009 22624 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:43:40.229436 22624 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:43:40.230432 22598 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 18:43:40.251433 22624 catalog_manager.cc:1347] Generated new cluster ID: 532796ba33b7483db4e04a2f999a9e0b
I20260811 18:43:40.251601 22624 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:43:40.269554 22624 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:43:40.271153 22624 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:43:40.280843 22624 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7: Generated new TSK 0
I20260811 18:43:40.281864 22624 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.185.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
I20260811 18:43:41.089736 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.185.148:41301/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41913
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:38877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:43:41 dist-test-slave-z8x9 krb5kdc[21657](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786473821, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 18:43:41.532706 22683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:43:41.533143 22683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:43:41.533394 22683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:43:41.545233 22683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:43:41.545392 22683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:43:41.545645 22683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:43:41.561636 22683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.185.148:41301/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:43:41.563618 22683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:43:41.565485 22683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:43:41.589738 22704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:41.590281 22706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:41.591185 22702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:43:41.593228 22683 server_base.cc:1034] running on GCE node
I20260811 18:43:41.593951 22683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:43:41.595034 22683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:43:41.596283 22683 hybrid_clock.cc:648] HybridClock initialized: now 1786473821596224 us; error 52 us; skew 500 ppm
I20260811 18:43:41.596630 22683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:43:41 dist-test-slave-z8x9 krb5kdc[21657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473821, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:43:41.600349 22683 init.cc:377] Logged in from keytab as kudu/127.30.185.129@KRBTEST.COM (short username kudu)
I20260811 18:43:41.602197 22683 webserver.cc:460] Webserver started at http://127.30.185.129:34307/ using document root <none> and password file <none>
I20260811 18:43:41.603140 22683 fs_manager.cc:359] Metadata directory not provided
I20260811 18:43:41.603302 22683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:43:41.603701 22683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:43:41 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473821, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
I20260811 18:43:41.624425 22683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "cb905b4ee7cf4a7591df70cf1eabba3f"
format_stamp: "Formatted at 2026-08-11 18:43:41 on dist-test-slave-z8x9"
server_key: "7fcfcad39c2079ed1bc28964041d7f96"
server_key_iv: "b33ff2f01921fa8e2254db74b925f88b"
server_key_version: "kuduclusterkey@0"
I20260811 18:43:41.625763 22683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "cb905b4ee7cf4a7591df70cf1eabba3f"
format_stamp: "Formatted at 2026-08-11 18:43:41 on dist-test-slave-z8x9"
server_key: "7fcfcad39c2079ed1bc28964041d7f96"
server_key_iv: "b33ff2f01921fa8e2254db74b925f88b"
server_key_version: "kuduclusterkey@0"
I20260811 18:43:41.659371 22683 fs_manager.cc:691] Time spent creating directory manager: real 0.033s user 0.006s sys 0.017s
I20260811 18:43:41.683177 22719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:43:41.685020 22683 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.005s sys 0.001s
I20260811 18:43:41.685256 22683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "cb905b4ee7cf4a7591df70cf1eabba3f"
format_stamp: "Formatted at 2026-08-11 18:43:41 on dist-test-slave-z8x9"
server_key: "7fcfcad39c2079ed1bc28964041d7f96"
server_key_iv: "b33ff2f01921fa8e2254db74b925f88b"
server_key_version: "kuduclusterkey@0"
I20260811 18:43:41.685534 22683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:43:41.703769 22683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:43:41.708235 22683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:43:41.708557 22683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:43:41.709677 22683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:43:41.711246 22683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:43:41.711373 22683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 18:43:41.711526 22683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:43:41.711616 22683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:43:41 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473821, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
I20260811 18:43:41.820729 22683 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:42885
I20260811 18:43:41.820757 22917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:42885 every 8 connection(s)
I20260811 18:43:41.822751 22683 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:43:41.829519 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 22683
I20260811 18:43:41.829802 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:43:41.830888 31462 external_mini_cluster.cc:1428] Setting key 55e5e0f9b60a53c731e8a34e2e3755bc
Aug 11 18:43:41 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473821, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.185.130@KRBTEST.COM; defaulting to no policy
I20260811 18:43:41.900373 22923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:41.829529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:53623 (local address 127.30.185.190:41913)
0811 18:43:41.869586 (+ 40057us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:41.869593 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:41.870549 (+ 956us) server_negotiation.cc:407] Connection header received
0811 18:43:41.876776 (+ 6227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:41.876783 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:41.876894 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:41.877038 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:41.879316 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:41.880617 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:41.883458 (+ 2841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:41.884257 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:41.884400 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:41.889492 (+ 5092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:41.889502 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:41.889509 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:41.889559 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:41.894060 (+ 4501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:41.895230 (+ 1170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:41.895236 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:41.895240 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:41.895416 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:41.895995 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:41.896000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:41.896003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:41.896461 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:41.896614 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:41.899751 (+ 3137us) server_negotiation.cc:299] Negotiation successful
0811 18:43:41.899997 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39912,"thread_start_us":39796,"threads_started":1}
I20260811 18:43:41.900905 22925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:41.830828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41913 (local address 127.30.185.129:53623)
0811 18:43:41.869768 (+ 38940us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:41.869864 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0811 18:43:41.873448 (+ 3584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:43:41.877408 (+ 3960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:43:41.877423 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:43:41.878184 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:43:41.879067 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:41.879097 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:41.880836 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:41.880843 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:41.883247 (+ 2404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:41.883269 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:41.884745 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:41.884755 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:41.885090 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:41.886042 (+ 952us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:43:41.886077 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:43:41.889208 (+ 3131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:43:41.894316 (+ 5108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:41.894329 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:41.894353 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:41.895060 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:41.895570 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:41.895575 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:41.895578 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:41.895843 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:41.896698 (+ 855us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:43:41.896706 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:43:41.897250 (+ 544us) client_negotiation.cc:769] Sending connection context
0811 18:43:41.897602 (+ 352us) client_negotiation.cc:240] Negotiation successful
0811 18:43:41.898203 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34648,"thread_start_us":34390,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.130@KRBTEST.COM" created.
I20260811 18:43:41.902954 22921 heartbeater.cc:347] Connected to a master server at 127.30.185.190:41913
I20260811 18:43:41.903330 22921 heartbeater.cc:464] Registering TS with master...
I20260811 18:43:41.904269 22921 heartbeater.cc:511] Master 127.30.185.190:41913 requested a full tablet report, sending...
I20260811 18:43:41.906893 22541 ts_manager.cc:194] Registered new tserver with Master: cb905b4ee7cf4a7591df70cf1eabba3f (127.30.185.129:42885)
I20260811 18:43:41.909106 22541 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:53623
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
W20260811 18:43:41.940063 31462 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.30.185.130: real 0.106s user 0.000s sys 0.030s
I20260811 18:43:41.940918 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.185.148:41301/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41913
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:38877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:43:42 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473821, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
W20260811 18:43:42.398241 22928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:43:42.398784 22928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:43:42.398998 22928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:43:42.410902 22928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:43:42.411233 22928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:43:42.411504 22928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:43:42.427429 22928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.185.148:41301/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:43:42.429627 22928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:43:42.431494 22928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:43:42.466317 22938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:42.470916 22937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:42.473711 22947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:43:42.474335 22928 server_base.cc:1034] running on GCE node
I20260811 18:43:42.475409 22928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:43:42.479888 22928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:43:42.481375 22928 hybrid_clock.cc:648] HybridClock initialized: now 1786473822481263 us; error 110 us; skew 500 ppm
I20260811 18:43:42.481839 22928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:43:42 dist-test-slave-z8x9 krb5kdc[21657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473822, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:43:42.486618 22928 init.cc:377] Logged in from keytab as kudu/127.30.185.130@KRBTEST.COM (short username kudu)
I20260811 18:43:42.507498 22928 webserver.cc:460] Webserver started at http://127.30.185.130:42963/ using document root <none> and password file <none>
I20260811 18:43:42.508796 22928 fs_manager.cc:359] Metadata directory not provided
I20260811 18:43:42.509064 22928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:43:42.509595 22928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:43:42 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473822, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
I20260811 18:43:42.534864 22928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "17a2f7dde30b4f2f85a67ca15c4fa88a"
format_stamp: "Formatted at 2026-08-11 18:43:42 on dist-test-slave-z8x9"
server_key: "c5d3d684a1f0bb66a384cae3a5bad651"
server_key_iv: "a6dbaee7dd76221d14e78e3dcf7b4bd9"
server_key_version: "kuduclusterkey@0"
I20260811 18:43:42.537895 22928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "17a2f7dde30b4f2f85a67ca15c4fa88a"
format_stamp: "Formatted at 2026-08-11 18:43:42 on dist-test-slave-z8x9"
server_key: "c5d3d684a1f0bb66a384cae3a5bad651"
server_key_iv: "a6dbaee7dd76221d14e78e3dcf7b4bd9"
server_key_version: "kuduclusterkey@0"
I20260811 18:43:42.567648 22928 fs_manager.cc:691] Time spent creating directory manager: real 0.029s user 0.004s sys 0.023s
I20260811 18:43:42.636024 22959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:43:42.638254 22928 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.001s sys 0.004s
I20260811 18:43:42.638470 22928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "17a2f7dde30b4f2f85a67ca15c4fa88a"
format_stamp: "Formatted at 2026-08-11 18:43:42 on dist-test-slave-z8x9"
server_key: "c5d3d684a1f0bb66a384cae3a5bad651"
server_key_iv: "a6dbaee7dd76221d14e78e3dcf7b4bd9"
server_key_version: "kuduclusterkey@0"
I20260811 18:43:42.638761 22928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:43:42.674300 22928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:43:42.679543 22928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:43:42.679903 22928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:43:42.681128 22928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:43:42.682993 22928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:43:42.683094 22928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:43:42.683218 22928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:43:42.683279 22928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:43:42.723789 22928 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:34473
I20260811 18:43:42.723824 23176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:34473 every 8 connection(s)
I20260811 18:43:42.725499 22928 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:43:42.731171 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 22928
I20260811 18:43:42.731348 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:43:42.731914 31462 external_mini_cluster.cc:1428] Setting key eff9fcae8bda914c89aee0c98f90fc7b
Aug 11 18:43:42 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473822, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:43:42.747357 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:42.728246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:48325 (local address 127.30.185.190:41913)
0811 18:43:42.728809 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:42.728816 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:42.729887 (+ 1071us) server_negotiation.cc:407] Connection header received
0811 18:43:42.732286 (+ 2399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:42.732292 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:42.732394 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:42.732520 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:42.734480 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:42.735226 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:42.736804 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:42.737296 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:42.737432 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:42.740931 (+ 3499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:42.740937 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:42.740940 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:42.740973 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:42.743855 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:42.744770 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:42.744779 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:42.744784 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:42.744916 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:42.745501 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:42.745509 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:42.745512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:42.745815 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:42.745988 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:42.746726 (+ 738us) server_negotiation.cc:299] Negotiation successful
0811 18:43:42.746961 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":258,"threads_started":1}
I20260811 18:43:42.748636 23186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:42.728854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41913 (local address 127.30.185.130:48325)
0811 18:43:42.729703 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:42.729750 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:43:42.731947 (+ 2197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:43:42.732813 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:43:42.732829 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:43:42.733480 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:43:42.734309 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:42.734330 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:42.735395 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:42.735399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:42.736625 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:42.736644 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:42.737534 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:42.737544 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:42.737779 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:42.738509 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:43:42.738537 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:43:42.740683 (+ 2146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:43:42.744009 (+ 3326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:42.744018 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:42.744038 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:42.744644 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:42.745029 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:42.745035 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:42.745039 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:42.745375 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:42.745949 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:43:42.745958 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:43:42.746337 (+ 379us) client_negotiation.cc:769] Sending connection context
0811 18:43:42.746619 (+ 282us) client_negotiation.cc:240] Negotiation successful
0811 18:43:42.747173 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":371,"threads_started":1}
I20260811 18:43:42.750469 23177 heartbeater.cc:347] Connected to a master server at 127.30.185.190:41913
I20260811 18:43:42.750816 23177 heartbeater.cc:464] Registering TS with master...
I20260811 18:43:42.751662 23177 heartbeater.cc:511] Master 127.30.185.190:41913 requested a full tablet report, sending...
I20260811 18:43:42.753458 22541 ts_manager.cc:194] Registered new tserver with Master: 17a2f7dde30b4f2f85a67ca15c4fa88a (127.30.185.130:34473)
I20260811 18:43:42.754766 22541 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:48325
WARNING: no policy specified for kudu/127.30.185.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
I20260811 18:43:42.805861 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.185.148:41301/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41913
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:38877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:43:42.913002 22921 heartbeater.cc:503] Master 127.30.185.190:41913 was elected leader, sending a full tablet report...
W20260811 18:43:43.171846 23191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:43:43.172135 23191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:43:43.172297 23191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:43:43.179811 23191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:43:43.179927 23191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:43:43.180094 23191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:43:43.190344 23191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.185.148:41301/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:41913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:43:43.191751 23191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:43:43.193097 23191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:43:43.206799 23196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:43.206840 23197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:43.206840 23205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:43:43.207731 23191 server_base.cc:1034] running on GCE node
I20260811 18:43:43.208472 23191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:43:43.209555 23191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:43:43.210845 23191 hybrid_clock.cc:648] HybridClock initialized: now 1786473823210802 us; error 44 us; skew 500 ppm
I20260811 18:43:43.211223 23191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:43:43 dist-test-slave-z8x9 krb5kdc[21657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473823, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:43:43.214639 23191 init.cc:377] Logged in from keytab as kudu/127.30.185.131@KRBTEST.COM (short username kudu)
I20260811 18:43:43.216215 23191 webserver.cc:460] Webserver started at http://127.30.185.131:34833/ using document root <none> and password file <none>
I20260811 18:43:43.216866 23191 fs_manager.cc:359] Metadata directory not provided
I20260811 18:43:43.216967 23191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:43:43.217414 23191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:43:43 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473823, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
I20260811 18:43:43.232475 23191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "49a0917360fb4c46b936d5d24bc72d08"
format_stamp: "Formatted at 2026-08-11 18:43:43 on dist-test-slave-z8x9"
server_key: "9fbadbe058f99395d751850164b2e37b"
server_key_iv: "cc35eff190af158d52b6ef85218d0598"
server_key_version: "kuduclusterkey@0"
I20260811 18:43:43.233529 23191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "49a0917360fb4c46b936d5d24bc72d08"
format_stamp: "Formatted at 2026-08-11 18:43:43 on dist-test-slave-z8x9"
server_key: "9fbadbe058f99395d751850164b2e37b"
server_key_iv: "cc35eff190af158d52b6ef85218d0598"
server_key_version: "kuduclusterkey@0"
I20260811 18:43:43.239535 23191 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 18:43:43.255934 23220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:43:43.257903 23191 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 18:43:43.258104 23191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "49a0917360fb4c46b936d5d24bc72d08"
format_stamp: "Formatted at 2026-08-11 18:43:43 on dist-test-slave-z8x9"
server_key: "9fbadbe058f99395d751850164b2e37b"
server_key_iv: "cc35eff190af158d52b6ef85218d0598"
server_key_version: "kuduclusterkey@0"
I20260811 18:43:43.258387 23191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:43:43.279712 23191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:43:43.283742 23191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:43:43.284011 23191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:43:43.284940 23191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:43:43.286386 23191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:43:43.286463 23191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:43:43.286551 23191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:43:43.286595 23191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:43:43.322341 23191 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:42663
I20260811 18:43:43.322400 23436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:42663 every 8 connection(s)
I20260811 18:43:43.323944 23191 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:43:43.332134 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 23191
I20260811 18:43:43.332365 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:43:43.332844 31462 external_mini_cluster.cc:1428] Setting key b590f1ca72d3b9bffd7baf2b4e98c951
Aug 11 18:43:43 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473823, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:43:43.343904 23448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:43.326562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:54123 (local address 127.30.185.190:41913)
0811 18:43:43.327085 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:43.327090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:43.327881 (+ 791us) server_negotiation.cc:407] Connection header received
0811 18:43:43.329776 (+ 1895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:43.329781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:43.329870 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:43.329985 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:43.331397 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:43.332134 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:43.333864 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:43.334336 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:43.334467 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:43.337927 (+ 3460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:43.337932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:43.337936 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:43.337980 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:43.340621 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:43.341534 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:43.341539 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:43.341541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:43.341674 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:43.342142 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:43.342146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:43.342148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:43.342440 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:43.342556 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:43.343279 (+ 723us) server_negotiation.cc:299] Negotiation successful
0811 18:43:43.343512 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":275,"threads_started":1}
I20260811 18:43:43.344771 23449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:43.327013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41913 (local address 127.30.185.131:54123)
0811 18:43:43.327718 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:43.327760 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:43:43.329501 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:43:43.330229 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:43:43.330237 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:43:43.330651 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:43:43.331224 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:43.331241 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:43.332288 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:43.332294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:43.333703 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:43.333728 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:43.334568 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:43.334575 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:43.334818 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:43.335557 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:43:43.335580 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:43:43.337716 (+ 2136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:43:43.340783 (+ 3067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:43.340791 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:43.340806 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:43.341388 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:43.341820 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:43.341823 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:43.341826 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:43.342038 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:43.342558 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:43:43.342564 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:43:43.342887 (+ 323us) client_negotiation.cc:769] Sending connection context
0811 18:43:43.343124 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 18:43:43.343547 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":290,"threads_started":1}
I20260811 18:43:43.346462 23442 heartbeater.cc:347] Connected to a master server at 127.30.185.190:41913
I20260811 18:43:43.346876 23442 heartbeater.cc:464] Registering TS with master...
I20260811 18:43:43.347843 23442 heartbeater.cc:511] Master 127.30.185.190:41913 requested a full tablet report, sending...
I20260811 18:43:43.349488 22541 ts_manager.cc:194] Registered new tserver with Master: 49a0917360fb4c46b936d5d24bc72d08 (127.30.185.131:42663)
I20260811 18:43:43.350453 22541 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:54123
I20260811 18:43:43.361430 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:43:43.363147 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 22683
Aug 11 18:43:43 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.129@KRBTEST.COM
I20260811 18:43:43.380378 23451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:43.364186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60822 (local address 127.30.185.129:42885)
0811 18:43:43.364871 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:43.364878 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:43.364905 (+ 27us) server_negotiation.cc:407] Connection header received
0811 18:43:43.364980 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:43.364986 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:43.365258 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:43.365400 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:43.366981 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:43.368067 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:43.369693 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:43.370250 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:43.370363 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:43.374789 (+ 4426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:43.374800 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:43.374821 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:43.374856 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:43.377784 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:43.378562 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:43.378569 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:43.378571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:43.378720 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:43.379057 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:43.379060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:43.379062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:43.379401 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:43.379634 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:43.379872 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 18:43:43.380074 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":371,"threads_started":1}
I20260811 18:43:43.381711 22908 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:43:43.467216 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 22683
I20260811 18:43:43.495990 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 22928
Aug 11 18:43:43 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.130@KRBTEST.COM
I20260811 18:43:43.517654 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:43.496915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53654 (local address 127.30.185.130:34473)
0811 18:43:43.497556 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:43.497564 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:43.497591 (+ 27us) server_negotiation.cc:407] Connection header received
0811 18:43:43.497673 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:43.497680 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:43.497893 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:43.498031 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:43.499356 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:43.500484 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:43.502064 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:43.502629 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:43.502730 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:43.511732 (+ 9002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:43.511743 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:43.511758 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:43.511806 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:43.514826 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:43.515518 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:43.515525 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:43.515527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:43.515669 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:43.516072 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:43.516076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:43.516078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:43.516440 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:43.516669 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:43.516947 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 18:43:43.517292 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":398,"threads_started":1}
I20260811 18:43:43.518932 23165 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:43:43.607354 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 22928
I20260811 18:43:43.629842 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 23191
Aug 11 18:43:43 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.131@KRBTEST.COM
I20260811 18:43:43.649237 23455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:43.631096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56814 (local address 127.30.185.131:42663)
0811 18:43:43.631839 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:43.631846 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:43.631875 (+ 29us) server_negotiation.cc:407] Connection header received
0811 18:43:43.631939 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:43.631946 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:43.632163 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:43.632354 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:43.633754 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:43.634761 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:43.636007 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:43.636599 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:43.636741 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:43.641767 (+ 5026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:43.641785 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:43.641810 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:43.641864 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:43.645981 (+ 4117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:43.646823 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:43.646836 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:43.646843 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:43.647026 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:43.647449 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:43.647462 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:43.647465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:43.647839 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:43.648223 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:43.648506 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 18:43:43.648796 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":423,"threads_started":1}
I20260811 18:43:43.650638 23430 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:43:43.735577 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 23191
I20260811 18:43:43.761055 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 22479
I20260811 18:43:43.762439 22591 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:43:43.873335 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 22479
I20260811 18:43:43.911495 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:41913
--webserver_interface=127.30.185.190
--webserver_port=42199
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:38877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:41913
--ranger_config_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.185.148:41301/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.185.148:41301/kms with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:43:44.255828 23459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:43:44.256147 23459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:43:44.256340 23459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:43:44.263623 23459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:43:44.263738 23459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:43:44.263868 23459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:43:44.263981 23459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:43:44.264076 23459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:43:44.274816 23459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.185.148:41301/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:41913
--ranger_config_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:41913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=42199
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:43:44.276258 23459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:43:44.277736 23459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:43:44.289798 23480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:44.289837 23479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:44.289885 23485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:43:44.291023 23459 server_base.cc:1034] running on GCE node
I20260811 18:43:44.291934 23459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:43:44.293499 23459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:43:44.294763 23459 hybrid_clock.cc:648] HybridClock initialized: now 1786473824294718 us; error 90 us; skew 500 ppm
I20260811 18:43:44.295147 23459 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:43:44 dist-test-slave-z8x9 krb5kdc[21657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473824, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:43:44.298609 23459 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:43:44.300223 23459 webserver.cc:460] Webserver started at http://127.30.185.190:42199/ using document root <none> and password file <none>
I20260811 18:43:44.300937 23459 fs_manager.cc:359] Metadata directory not provided
I20260811 18:43:44.301085 23459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 18:43:45 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473824, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
I20260811 18:43:45.264040 23459 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:43:45.267357 23503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:43:45.268889 23459 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:43:45.269094 23459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "b04b3907879f447ab707149cb088f8b7"
format_stamp: "Formatted at 2026-08-11 18:43:40 on dist-test-slave-z8x9"
server_key: "94c9ed62be5aa551c8fe35b6e062c57f"
server_key_iv: "53701a5e9a149999753e628f5bcf024f"
server_key_version: "kuduclusterkey@0"
I20260811 18:43:45.270169 23459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:43:45.293085 23459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:43:45.297906 23459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:43:45.298211 23459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:43:45.317806 23459 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:41913
I20260811 18:43:45.317847 23595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:41913 every 8 connection(s)
I20260811 18:43:45.319311 23459 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:43:45.322094 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 23459
I20260811 18:43:45.323212 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.185.148:41301/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:42885
--local_ip_for_outbound_sockets=127.30.185.129
--tserver_master_addrs=127.30.185.190:41913
--webserver_port=34307
--webserver_interface=127.30.185.129
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:38877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.185.148:41301/kms with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:43:45.328701 23596 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:43:45.334059 23596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7: Bootstrap starting.
I20260811 18:43:45.353529 23596 log.cc:824] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7: Log is configured to *not* fsync() on all Append() calls
I20260811 18:43:45.362643 23596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:43:45.363185 23596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7: Bootstrap complete.
I20260811 18:43:45.372347 23596 raft_consensus.cc:348] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b04b3907879f447ab707149cb088f8b7" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41913 } }
I20260811 18:43:45.373306 23596 raft_consensus.cc:729] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b04b3907879f447ab707149cb088f8b7, State: Initialized, Role: FOLLOWER
I20260811 18:43:45.374066 23596 consensus_queue.cc:260] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b04b3907879f447ab707149cb088f8b7" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41913 } }
I20260811 18:43:45.374356 23596 raft_consensus.cc:388] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:43:45.374483 23596 raft_consensus.cc:482] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:43:45.374698 23596 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:43:45.378932 23596 raft_consensus.cc:504] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b04b3907879f447ab707149cb088f8b7" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41913 } }
I20260811 18:43:45.379465 23596 leader_election.cc:302] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b04b3907879f447ab707149cb088f8b7; no voters:
I20260811 18:43:45.380143 23596 leader_election.cc:288] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 18:43:45.380299 23604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:43:45.381594 23604 raft_consensus.cc:686] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [term 2 LEADER]: Becoming Leader. State: Replica: b04b3907879f447ab707149cb088f8b7, State: Running, Role: LEADER
I20260811 18:43:45.382270 23604 consensus_queue.cc:237] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b04b3907879f447ab707149cb088f8b7" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41913 } }
I20260811 18:43:45.383159 23596 sys_catalog.cc:564] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:43:45.385594 23607 sys_catalog.cc:455] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b04b3907879f447ab707149cb088f8b7. Latest consensus state: current_term: 2 leader_uuid: "b04b3907879f447ab707149cb088f8b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b04b3907879f447ab707149cb088f8b7" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41913 } } }
I20260811 18:43:45.386003 23607 sys_catalog.cc:458] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [sys.catalog]: This master's current role is: LEADER
I20260811 18:43:45.386602 23605 sys_catalog.cc:455] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b04b3907879f447ab707149cb088f8b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b04b3907879f447ab707149cb088f8b7" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 41913 } } }
I20260811 18:43:45.386891 23605 sys_catalog.cc:458] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7 [sys.catalog]: This master's current role is: LEADER
I20260811 18:43:45.392215 23610 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:43:45.406111 23610 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:43:45.407672 23610 catalog_manager.cc:1259] Loaded cluster ID: 532796ba33b7483db4e04a2f999a9e0b
I20260811 18:43:45.407871 23610 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:43:45.422513 23610 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:43:45.424582 23610 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7: Loaded TSK: 0
I20260811 18:43:45.426560 23610 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:43:45.444059 23596 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260811 18:43:45.806298 23601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:43:45.806726 23601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:43:45.806890 23601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:43:45.819053 23601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:43:45.819226 23601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:43:45.819433 23601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:43:45.835487 23601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.185.148:41301/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:42885
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=34307
--tserver_master_addrs=127.30.185.190:41913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:43:45.837287 23601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:43:45.839057 23601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:43:45.862972 23653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:45.864089 23654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:45.873004 23662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:43:45.876540 23601 server_base.cc:1034] running on GCE node
I20260811 18:43:45.877552 23601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:43:45.878947 23601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:43:45.882335 23601 hybrid_clock.cc:648] HybridClock initialized: now 1786473825882259 us; error 73 us; skew 500 ppm
I20260811 18:43:45.882771 23601 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:43:45 dist-test-slave-z8x9 krb5kdc[21657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473825, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:43:45.891302 23601 init.cc:377] Logged in from keytab as kudu/127.30.185.129@KRBTEST.COM (short username kudu)
I20260811 18:43:45.894964 23601 webserver.cc:460] Webserver started at http://127.30.185.129:34307/ using document root <none> and password file <none>
I20260811 18:43:45.896044 23601 fs_manager.cc:359] Metadata directory not provided
I20260811 18:43:45.896239 23601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 18:43:45 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473825, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
I20260811 18:43:45.967918 23601 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.003s sys 0.004s
I20260811 18:43:45.977953 23676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:43:45.980023 23601 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.005s sys 0.003s
I20260811 18:43:45.980228 23601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "cb905b4ee7cf4a7591df70cf1eabba3f"
format_stamp: "Formatted at 2026-08-11 18:43:41 on dist-test-slave-z8x9"
server_key: "7fcfcad39c2079ed1bc28964041d7f96"
server_key_iv: "b33ff2f01921fa8e2254db74b925f88b"
server_key_version: "kuduclusterkey@0"
I20260811 18:43:45.981808 23601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:43:46.032263 23601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:43:46.036899 23601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:43:46.037263 23601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:43:46.038443 23601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:43:46.040849 23601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:43:46.040977 23601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:43:46.041095 23601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:43:46.041208 23601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:43:46.085951 23601 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:42885
I20260811 18:43:46.085984 23882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:42885 every 8 connection(s)
I20260811 18:43:46.087862 23601 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:43:46.088848 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 23601
I20260811 18:43:46.089929 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.185.148:41301/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:34473
--local_ip_for_outbound_sockets=127.30.185.130
--tserver_master_addrs=127.30.185.190:41913
--webserver_port=42963
--webserver_interface=127.30.185.130
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:38877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.185.148:41301/kms with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:43:46 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473825, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:43:46.151461 23890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:46.097973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:38403 (local address 127.30.185.190:41913)
0811 18:43:46.115645 (+ 17672us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:46.115671 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:46.119543 (+ 3872us) server_negotiation.cc:407] Connection header received
0811 18:43:46.122946 (+ 3403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:46.122970 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:46.123547 (+ 577us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:46.124029 (+ 482us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:46.126900 (+ 2871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:46.129517 (+ 2617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:46.132655 (+ 3138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:46.134111 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:46.134341 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:46.140455 (+ 6114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:46.140467 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:46.140490 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:46.140544 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:46.144988 (+ 4444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:46.146247 (+ 1259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:46.146258 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:46.146267 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:46.146523 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:46.147147 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:46.147153 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:46.147157 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:46.147928 (+ 771us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:46.148134 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:46.149395 (+ 1261us) server_negotiation.cc:299] Negotiation successful
0811 18:43:46.149885 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17278,"thread_start_us":17811,"threads_started":1}
I20260811 18:43:46.154788 23889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:46.092507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41913 (local address 127.30.185.129:38403)
0811 18:43:46.119286 (+ 26779us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:46.119362 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0811 18:43:46.122090 (+ 2728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:43:46.124832 (+ 2742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:43:46.124847 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:43:46.125734 (+ 887us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:43:46.126626 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:46.126656 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:46.129769 (+ 3113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:46.129778 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:46.132436 (+ 2658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:46.132457 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:46.135009 (+ 2552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:46.135019 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:46.135424 (+ 405us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:46.136449 (+ 1025us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:43:46.136493 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:43:46.140064 (+ 3571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:43:46.145317 (+ 5253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:46.145328 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:46.145346 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:46.146027 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:46.146676 (+ 649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:46.146682 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:46.146687 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:46.146988 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:46.148236 (+ 1248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:43:46.148246 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:43:46.148725 (+ 479us) client_negotiation.cc:769] Sending connection context
0811 18:43:46.150557 (+ 1832us) client_negotiation.cc:240] Negotiation successful
0811 18:43:46.153404 (+ 2847us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26336,"thread_start_us":26054,"threads_started":1}
I20260811 18:43:46.158892 23883 heartbeater.cc:347] Connected to a master server at 127.30.185.190:41913
I20260811 18:43:46.159286 23883 heartbeater.cc:464] Registering TS with master...
I20260811 18:43:46.160248 23883 heartbeater.cc:511] Master 127.30.185.190:41913 requested a full tablet report, sending...
W20260811 18:43:46.164134 23883 heartbeater.cc:644] Failed to heartbeat to 127.30.185.190:41913 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 18:43:46.164402 23883 heartbeater.cc:464] Registering TS with master...
I20260811 18:43:46.164733 23883 heartbeater.cc:511] Master 127.30.185.190:41913 requested a full tablet report, sending...
I20260811 18:43:46.166127 23883 heartbeater.cc:464] Registering TS with master...
I20260811 18:43:46.166431 23883 heartbeater.cc:511] Master 127.30.185.190:41913 requested a full tablet report, sending...
W20260811 18:43:46.167663 23883 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:43:46.539904 23892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:43:46.540261 23892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:43:46.540418 23892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:43:46.551252 23892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:43:46.551411 23892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:43:46.551651 23892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:43:46.564273 23892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.185.148:41301/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:34473
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=42963
--tserver_master_addrs=127.30.185.190:41913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:43:46.566164 23892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:43:46.567590 23892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:43:46.612458 23911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:46.618213 23912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:43:46.619105 23892 server_base.cc:1034] running on GCE node
W20260811 18:43:46.625492 23925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:43:46.626471 23892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:43:46.641558 23892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:43:46.642972 23892 hybrid_clock.cc:648] HybridClock initialized: now 1786473826642860 us; error 77 us; skew 500 ppm
I20260811 18:43:46.643527 23892 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:43:46 dist-test-slave-z8x9 krb5kdc[21657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473826, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:43:46.647179 23892 init.cc:377] Logged in from keytab as kudu/127.30.185.130@KRBTEST.COM (short username kudu)
I20260811 18:43:46.649209 23892 webserver.cc:460] Webserver started at http://127.30.185.130:42963/ using document root <none> and password file <none>
I20260811 18:43:46.649976 23892 fs_manager.cc:359] Metadata directory not provided
I20260811 18:43:46.650151 23892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 18:43:46 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473826, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
I20260811 18:43:46.692947 23892 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.003s sys 0.007s
Aug 11 18:43:46 dist-test-slave-z8x9 krb5kdc[21657](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786473826, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:43:46.703452 23934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:43:46.705505 23892 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.003s sys 0.007s
I20260811 18:43:46.705713 23892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "17a2f7dde30b4f2f85a67ca15c4fa88a"
format_stamp: "Formatted at 2026-08-11 18:43:42 on dist-test-slave-z8x9"
server_key: "c5d3d684a1f0bb66a384cae3a5bad651"
server_key_iv: "a6dbaee7dd76221d14e78e3dcf7b4bd9"
server_key_version: "kuduclusterkey@0"
I20260811 18:43:46.707195 23892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:43:46.764899 23892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:43:46.770409 23892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:43:46.770779 23892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:43:46.778225 23892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:43:46.787665 23892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:43:46.787806 23892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:43:46.787927 23892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:43:46.788002 23892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:43:46.833436 23892 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:34473
I20260811 18:43:46.833503 24141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:34473 every 8 connection(s)
I20260811 18:43:46.835389 23892 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:43:46.840981 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 23892
I20260811 18:43:46.841845 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.185.148:41301/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:42663
--local_ip_for_outbound_sockets=127.30.185.131
--tserver_master_addrs=127.30.185.190:41913
--webserver_port=34833
--webserver_interface=127.30.185.131
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:38877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.185.148:41301/kms with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:43:46 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473826, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:43:46.909376 24153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:46.858578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:41029 (local address 127.30.185.190:41913)
0811 18:43:46.877723 (+ 19145us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:46.877733 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:46.879633 (+ 1900us) server_negotiation.cc:407] Connection header received
0811 18:43:46.886801 (+ 7168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:46.886809 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:46.886937 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:46.887057 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:46.889716 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:46.891406 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:46.894197 (+ 2791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:46.894962 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:46.895078 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:46.900222 (+ 5144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:46.900232 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:46.900240 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:46.900291 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:46.904665 (+ 4374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:46.905868 (+ 1203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:46.905876 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:46.905880 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:46.906064 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:46.906726 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:46.906733 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:46.906739 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:46.907194 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:46.907354 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:46.908556 (+ 1202us) server_negotiation.cc:299] Negotiation successful
0811 18:43:46.908839 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18956,"thread_start_us":18831,"threads_started":1}
I20260811 18:43:46.912158 24149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:46.857840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41913 (local address 127.30.185.130:41029)
0811 18:43:46.879055 (+ 21215us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:46.879149 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0811 18:43:46.886428 (+ 7279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:43:46.887664 (+ 1236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:43:46.887680 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:43:46.888411 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:43:46.889487 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:46.889518 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:46.891653 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:46.891662 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:46.893991 (+ 2329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:46.894013 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:46.895483 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:46.895496 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:46.895861 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:46.896787 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:43:46.896829 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:43:46.899929 (+ 3100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:43:46.904883 (+ 4954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:46.904896 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:46.904919 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:46.905682 (+ 763us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:46.906242 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:46.906249 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:46.906255 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:46.906566 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:46.907465 (+ 899us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:43:46.907477 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:43:46.908036 (+ 559us) client_negotiation.cc:769] Sending connection context
0811 18:43:46.909886 (+ 1850us) client_negotiation.cc:240] Negotiation successful
0811 18:43:46.910566 (+ 680us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20191,"thread_start_us":19941,"threads_started":1}
I20260811 18:43:46.914119 24145 heartbeater.cc:347] Connected to a master server at 127.30.185.190:41913
I20260811 18:43:46.914480 24145 heartbeater.cc:464] Registering TS with master...
I20260811 18:43:46.915253 24145 heartbeater.cc:511] Master 127.30.185.190:41913 requested a full tablet report, sending...
I20260811 18:43:46.917358 23539 ts_manager.cc:194] Registered new tserver with Master: 17a2f7dde30b4f2f85a67ca15c4fa88a (127.30.185.130:34473)
I20260811 18:43:46.919829 23539 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:41029
I20260811 18:43:47.167846 23883 heartbeater.cc:464] Registering TS with master...
I20260811 18:43:47.168313 23883 heartbeater.cc:511] Master 127.30.185.190:41913 requested a full tablet report, sending...
I20260811 18:43:47.169942 23539 ts_manager.cc:194] Registered new tserver with Master: cb905b4ee7cf4a7591df70cf1eabba3f (127.30.185.129:42885)
I20260811 18:43:47.171324 23539 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:38403
Aug 11 18:43:47 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473826, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
W20260811 18:43:47.336216 24154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:43:47.336664 24154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:43:47.336834 24154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:43:47.348272 24154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:43:47.348449 24154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:43:47.348654 24154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:43:47.364754 24154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.185.148:41301/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:42663
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=34833
--tserver_master_addrs=127.30.185.190:41913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:43:47.366658 24154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:43:47.368435 24154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:43:47.404031 24161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:47.404740 24159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:47.411397 24166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:43:47.413319 24154 server_base.cc:1034] running on GCE node
I20260811 18:43:47.414309 24154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:43:47.415529 24154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:43:47.416752 24154 hybrid_clock.cc:648] HybridClock initialized: now 1786473827416676 us; error 64 us; skew 500 ppm
I20260811 18:43:47.417096 24154 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:43:47 dist-test-slave-z8x9 krb5kdc[21657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473827, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:43:47.420539 24154 init.cc:377] Logged in from keytab as kudu/127.30.185.131@KRBTEST.COM (short username kudu)
I20260811 18:43:47.425578 24154 webserver.cc:460] Webserver started at http://127.30.185.131:34833/ using document root <none> and password file <none>
I20260811 18:43:47.426338 24154 fs_manager.cc:359] Metadata directory not provided
I20260811 18:43:47.426512 24154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 18:43:47 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473827, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
I20260811 18:43:47.448421 24154 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:43:47.451781 24184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:43:47.453469 24154 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:43:47.453679 24154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "49a0917360fb4c46b936d5d24bc72d08"
format_stamp: "Formatted at 2026-08-11 18:43:43 on dist-test-slave-z8x9"
server_key: "9fbadbe058f99395d751850164b2e37b"
server_key_iv: "cc35eff190af158d52b6ef85218d0598"
server_key_version: "kuduclusterkey@0"
I20260811 18:43:47.454862 24154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:43:47.478403 24154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:43:47.483260 24154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:43:47.483614 24154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:43:47.487860 24154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:43:47.493422 24154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:43:47.493556 24154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:43:47.493714 24154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:43:47.493808 24154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:43:47.558943 24154 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:42663
I20260811 18:43:47.559026 24378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:42663 every 8 connection(s)
I20260811 18:43:47.561103 24154 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:43:47.566445 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 24154
Aug 11 18:43:47 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473826, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for HTTP/127.30.185.148@KRBTEST.COM
Aug 11 18:43:47 dist-test-slave-z8x9 krb5kdc[21657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473827, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:43:47.640103 24385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:47.567531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53082 (local address 127.30.185.190:41913)
0811 18:43:47.613655 (+ 46124us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:47.613661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:47.616507 (+ 2846us) server_negotiation.cc:407] Connection header received
0811 18:43:47.616816 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:47.616823 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:47.616941 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:47.617058 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:47.621533 (+ 4475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.622804 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.625289 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.625969 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.626168 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:47.630289 (+ 4121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:47.630296 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:47.630301 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:47.630345 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:47.634357 (+ 4012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:47.635518 (+ 1161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:47.635524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:47.635527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:47.635703 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:47.637305 (+ 1602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:47.637311 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:47.637314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:47.637721 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:47.638031 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:47.639447 (+ 1416us) server_negotiation.cc:299] Negotiation successful
0811 18:43:47.639695 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45994,"thread_start_us":41404,"threads_started":1}
I20260811 18:43:47.642666 24384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:47.564996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:41913 (local address 127.30.185.131:34255)
0811 18:43:47.611238 (+ 46242us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:47.611319 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0811 18:43:47.615556 (+ 4237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:43:47.620176 (+ 4620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:43:47.620190 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:43:47.620896 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:43:47.623839 (+ 2943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:47.623866 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.626366 (+ 2500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.626371 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:47.628563 (+ 2192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:47.628602 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.629641 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.629650 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:47.629964 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:47.630788 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:43:47.630812 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:43:47.633287 (+ 2475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:43:47.637526 (+ 4239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:47.637537 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:47.637557 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:47.638266 (+ 709us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:47.638723 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:47.638728 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:47.638731 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:47.639020 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:47.639747 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:43:47.639756 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:43:47.640205 (+ 449us) client_negotiation.cc:769] Sending connection context
0811 18:43:47.640525 (+ 320us) client_negotiation.cc:240] Negotiation successful
0811 18:43:47.641119 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45853,"thread_start_us":48579,"threads_started":1}
I20260811 18:43:47.643810 24386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:47.565985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:34255 (local address 127.30.185.190:41913)
0811 18:43:47.608907 (+ 42922us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:47.608914 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:47.613232 (+ 4318us) server_negotiation.cc:407] Connection header received
0811 18:43:47.619393 (+ 6161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:47.619399 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:47.619480 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:47.619553 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:47.624083 (+ 4530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.625322 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.628760 (+ 3438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.629373 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.629519 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:47.633611 (+ 4092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:47.633619 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:47.633624 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:47.633680 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:47.637350 (+ 3670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:47.638416 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:47.638421 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:47.638424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:47.638569 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:47.639169 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:47.639175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:47.639178 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:47.639505 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:47.639638 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:47.643222 (+ 3584us) server_negotiation.cc:299] Negotiation successful
0811 18:43:47.643445 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42760,"thread_start_us":49853,"threads_started":1}
I20260811 18:43:47.645033 24379 heartbeater.cc:347] Connected to a master server at 127.30.185.190:41913
I20260811 18:43:47.645431 24379 heartbeater.cc:464] Registering TS with master...
I20260811 18:43:47.646404 24379 heartbeater.cc:511] Master 127.30.185.190:41913 requested a full tablet report, sending...
I20260811 18:43:47.648259 23539 ts_manager.cc:194] Registered new tserver with Master: 49a0917360fb4c46b936d5d24bc72d08 (127.30.185.131:42663)
I20260811 18:43:47.649091 23539 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:34255
I20260811 18:43:47.655943 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:43:47.677284 24386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:47.659529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53084 (local address 127.30.185.190:41913)
0811 18:43:47.659722 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:47.659730 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:47.660325 (+ 595us) server_negotiation.cc:407] Connection header received
0811 18:43:47.660536 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:47.660543 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:47.660628 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:47.660742 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:47.662777 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.663569 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.665402 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.665958 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.666085 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:47.669958 (+ 3873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:47.669965 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:47.669970 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:47.670016 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:47.673993 (+ 3977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:47.674917 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:47.674924 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:47.674928 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:47.675088 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:47.675554 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:47.675559 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:47.675563 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:47.675972 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:47.676150 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:47.676644 (+ 494us) server_negotiation.cc:299] Negotiation successful
0811 18:43:47.676875 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 18:43:47.682910 23539 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:43:47.695457 23539 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:43:47.758075 24410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:47.729028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56820 (local address 127.30.185.131:42663)
0811 18:43:47.748628 (+ 19600us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:47.748635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:47.748661 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:43:47.748733 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:47.748738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:47.748895 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:47.749104 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:43:47.749823 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.751472 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.754223 (+ 2751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.755827 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.756069 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:47.756831 (+ 762us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:47.757427 (+ 596us) server_negotiation.cc:299] Negotiation successful
0811 18:43:47.757612 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19434,"thread_start_us":19269,"threads_started":1}
I20260811 18:43:47.758839 24409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:47.728487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60836 (local address 127.30.185.129:42885)
0811 18:43:47.747109 (+ 18622us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:47.747118 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:47.747157 (+ 39us) server_negotiation.cc:407] Connection header received
0811 18:43:47.747303 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:47.747315 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:47.747646 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:47.747896 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:43:47.748585 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.750765 (+ 2180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.755130 (+ 4365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.756664 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.756846 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:47.757025 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:47.758275 (+ 1250us) server_negotiation.cc:299] Negotiation successful
0811 18:43:47.758448 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18394,"thread_start_us":18202,"threads_started":1}
I20260811 18:43:47.758131 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:47.728156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:42885 (local address 127.0.0.1:60836)
0811 18:43:47.746785 (+ 18629us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:47.746805 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:43:47.746939 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:43:47.747903 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:43:47.747908 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:43:47.747933 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:43:47.748384 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:47.748400 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.750957 (+ 2557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.750964 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:47.754907 (+ 3943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:47.754918 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.757265 (+ 2347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.757271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:47.757444 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:47.757474 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 18:43:47.757563 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 18:43:47.757704 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11493,"mutex_wait_us":23,"thread_start_us":11732,"threads_started":1}
I20260811 18:43:47.759830 24408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:47.728198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:42663 (local address 127.0.0.1:56820)
0811 18:43:47.739994 (+ 11796us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:47.740052 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 18:43:47.740240 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:43:47.749092 (+ 8852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:43:47.749097 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:43:47.749123 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:43:47.749589 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:47.749603 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.751663 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.751669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:47.754026 (+ 2357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:47.754042 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.756393 (+ 2351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.756399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:47.756597 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:47.756655 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 18:43:47.757036 (+ 381us) client_negotiation.cc:240] Negotiation successful
0811 18:43:47.758454 (+ 1418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11532,"mutex_wait_us":3,"thread_start_us":11407,"threads_started":1}
I20260811 18:43:47.760018 24412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:47.740942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53660 (local address 127.30.185.130:34473)
0811 18:43:47.746448 (+ 5506us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:47.746462 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:47.751331 (+ 4869us) server_negotiation.cc:407] Connection header received
0811 18:43:47.751808 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:47.751819 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:47.752034 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:47.752203 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:43:47.753144 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.754644 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.757083 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.758589 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.758731 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:47.758891 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:47.759534 (+ 643us) server_negotiation.cc:299] Negotiation successful
0811 18:43:47.759713 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5260,"thread_start_us":5110,"threads_started":1}
I20260811 18:43:47.763865 23764 tablet_service.cc:1467] Processing CreateTablet for tablet 2d1d5f0630a14484859a4e4e23a44bb7 (DEFAULT_TABLE table=test-table [id=e990ed07d91c43ce956347263ea542e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:43:47.763865 24264 tablet_service.cc:1467] Processing CreateTablet for tablet 2d1d5f0630a14484859a4e4e23a44bb7 (DEFAULT_TABLE table=test-table [id=e990ed07d91c43ce956347263ea542e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:43:47.765511 24264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d1d5f0630a14484859a4e4e23a44bb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:43:47.765530 23764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d1d5f0630a14484859a4e4e23a44bb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:43:47.771994 24413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:47.746533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:34473 (local address 127.0.0.1:53660)
0811 18:43:47.751180 (+ 4647us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:47.751217 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 18:43:47.751588 (+ 371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:43:47.752526 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:43:47.752530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:43:47.752554 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:43:47.752942 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:47.752952 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.754840 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.754846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:47.756911 (+ 2065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:47.756925 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.759096 (+ 2171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.759105 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:47.759309 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:47.759339 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 18:43:47.760285 (+ 946us) client_negotiation.cc:240] Negotiation successful
0811 18:43:47.760419 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4488,"thread_start_us":4813,"threads_started":1}
I20260811 18:43:47.776188 24026 tablet_service.cc:1467] Processing CreateTablet for tablet 2d1d5f0630a14484859a4e4e23a44bb7 (DEFAULT_TABLE table=test-table [id=e990ed07d91c43ce956347263ea542e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:43:47.777736 24026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d1d5f0630a14484859a4e4e23a44bb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:43:47.803498 24414 tablet_bootstrap.cc:492] T 2d1d5f0630a14484859a4e4e23a44bb7 P 49a0917360fb4c46b936d5d24bc72d08: Bootstrap starting.
I20260811 18:43:47.804850 24416 tablet_bootstrap.cc:492] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f: Bootstrap starting.
I20260811 18:43:47.808089 24414 tablet_bootstrap.cc:654] T 2d1d5f0630a14484859a4e4e23a44bb7 P 49a0917360fb4c46b936d5d24bc72d08: Neither blocks nor log segments found. Creating new log.
I20260811 18:43:47.809993 24417 tablet_bootstrap.cc:492] T 2d1d5f0630a14484859a4e4e23a44bb7 P 17a2f7dde30b4f2f85a67ca15c4fa88a: Bootstrap starting.
I20260811 18:43:47.810386 24414 log.cc:824] T 2d1d5f0630a14484859a4e4e23a44bb7 P 49a0917360fb4c46b936d5d24bc72d08: Log is configured to *not* fsync() on all Append() calls
I20260811 18:43:47.812337 24416 tablet_bootstrap.cc:654] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f: Neither blocks nor log segments found. Creating new log.
I20260811 18:43:47.814913 24416 log.cc:824] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:43:47.816587 24417 tablet_bootstrap.cc:654] T 2d1d5f0630a14484859a4e4e23a44bb7 P 17a2f7dde30b4f2f85a67ca15c4fa88a: Neither blocks nor log segments found. Creating new log.
I20260811 18:43:47.818300 24417 log.cc:824] T 2d1d5f0630a14484859a4e4e23a44bb7 P 17a2f7dde30b4f2f85a67ca15c4fa88a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:43:47.829900 24416 tablet_bootstrap.cc:492] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f: No bootstrap required, opened a new log
I20260811 18:43:47.829967 24417 tablet_bootstrap.cc:492] T 2d1d5f0630a14484859a4e4e23a44bb7 P 17a2f7dde30b4f2f85a67ca15c4fa88a: No bootstrap required, opened a new log
I20260811 18:43:47.830350 24416 ts_tablet_manager.cc:1397] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f: Time spent bootstrapping tablet: real 0.026s user 0.010s sys 0.000s
I20260811 18:43:47.830371 24417 ts_tablet_manager.cc:1397] T 2d1d5f0630a14484859a4e4e23a44bb7 P 17a2f7dde30b4f2f85a67ca15c4fa88a: Time spent bootstrapping tablet: real 0.021s user 0.011s sys 0.005s
I20260811 18:43:47.840639 24414 tablet_bootstrap.cc:492] T 2d1d5f0630a14484859a4e4e23a44bb7 P 49a0917360fb4c46b936d5d24bc72d08: No bootstrap required, opened a new log
I20260811 18:43:47.841002 24414 ts_tablet_manager.cc:1397] T 2d1d5f0630a14484859a4e4e23a44bb7 P 49a0917360fb4c46b936d5d24bc72d08: Time spent bootstrapping tablet: real 0.038s user 0.014s sys 0.015s
I20260811 18:43:47.851994 24414 raft_consensus.cc:348] T 2d1d5f0630a14484859a4e4e23a44bb7 P 49a0917360fb4c46b936d5d24bc72d08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a0917360fb4c46b936d5d24bc72d08" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42663 } } peers { permanent_uuid: "cb905b4ee7cf4a7591df70cf1eabba3f" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 42885 } } peers { permanent_uuid: "17a2f7dde30b4f2f85a67ca15c4fa88a" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 34473 } }
I20260811 18:43:47.852524 24414 raft_consensus.cc:374] T 2d1d5f0630a14484859a4e4e23a44bb7 P 49a0917360fb4c46b936d5d24bc72d08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:43:47.852619 24414 raft_consensus.cc:729] T 2d1d5f0630a14484859a4e4e23a44bb7 P 49a0917360fb4c46b936d5d24bc72d08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49a0917360fb4c46b936d5d24bc72d08, State: Initialized, Role: FOLLOWER
I20260811 18:43:47.853303 24414 consensus_queue.cc:260] T 2d1d5f0630a14484859a4e4e23a44bb7 P 49a0917360fb4c46b936d5d24bc72d08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a0917360fb4c46b936d5d24bc72d08" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42663 } } peers { permanent_uuid: "cb905b4ee7cf4a7591df70cf1eabba3f" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 42885 } } peers { permanent_uuid: "17a2f7dde30b4f2f85a67ca15c4fa88a" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 34473 } }
I20260811 18:43:47.857666 24416 raft_consensus.cc:348] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a0917360fb4c46b936d5d24bc72d08" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42663 } } peers { permanent_uuid: "cb905b4ee7cf4a7591df70cf1eabba3f" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 42885 } } peers { permanent_uuid: "17a2f7dde30b4f2f85a67ca15c4fa88a" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 34473 } }
I20260811 18:43:47.858278 24416 raft_consensus.cc:374] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:43:47.858474 24416 raft_consensus.cc:729] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb905b4ee7cf4a7591df70cf1eabba3f, State: Initialized, Role: FOLLOWER
I20260811 18:43:47.859447 24416 consensus_queue.cc:260] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a0917360fb4c46b936d5d24bc72d08" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42663 } } peers { permanent_uuid: "cb905b4ee7cf4a7591df70cf1eabba3f" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 42885 } } peers { permanent_uuid: "17a2f7dde30b4f2f85a67ca15c4fa88a" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 34473 } }
I20260811 18:43:47.861939 24414 ts_tablet_manager.cc:1428] T 2d1d5f0630a14484859a4e4e23a44bb7 P 49a0917360fb4c46b936d5d24bc72d08: Time spent starting tablet: real 0.021s user 0.021s sys 0.001s
I20260811 18:43:47.862304 24379 heartbeater.cc:503] Master 127.30.185.190:41913 was elected leader, sending a full tablet report...
I20260811 18:43:47.865054 24417 raft_consensus.cc:348] T 2d1d5f0630a14484859a4e4e23a44bb7 P 17a2f7dde30b4f2f85a67ca15c4fa88a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a0917360fb4c46b936d5d24bc72d08" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42663 } } peers { permanent_uuid: "cb905b4ee7cf4a7591df70cf1eabba3f" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 42885 } } peers { permanent_uuid: "17a2f7dde30b4f2f85a67ca15c4fa88a" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 34473 } }
I20260811 18:43:47.865573 24417 raft_consensus.cc:374] T 2d1d5f0630a14484859a4e4e23a44bb7 P 17a2f7dde30b4f2f85a67ca15c4fa88a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:43:47.865695 24417 raft_consensus.cc:729] T 2d1d5f0630a14484859a4e4e23a44bb7 P 17a2f7dde30b4f2f85a67ca15c4fa88a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17a2f7dde30b4f2f85a67ca15c4fa88a, State: Initialized, Role: FOLLOWER
I20260811 18:43:47.866451 24417 consensus_queue.cc:260] T 2d1d5f0630a14484859a4e4e23a44bb7 P 17a2f7dde30b4f2f85a67ca15c4fa88a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a0917360fb4c46b936d5d24bc72d08" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42663 } } peers { permanent_uuid: "cb905b4ee7cf4a7591df70cf1eabba3f" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 42885 } } peers { permanent_uuid: "17a2f7dde30b4f2f85a67ca15c4fa88a" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 34473 } }
I20260811 18:43:47.868332 24416 ts_tablet_manager.cc:1428] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f: Time spent starting tablet: real 0.038s user 0.013s sys 0.004s
I20260811 18:43:47.868724 23883 heartbeater.cc:503] Master 127.30.185.190:41913 was elected leader, sending a full tablet report...
I20260811 18:43:47.869856 24417 ts_tablet_manager.cc:1428] T 2d1d5f0630a14484859a4e4e23a44bb7 P 17a2f7dde30b4f2f85a67ca15c4fa88a: Time spent starting tablet: real 0.039s user 0.016s sys 0.000s
I20260811 18:43:47.870093 24145 heartbeater.cc:503] Master 127.30.185.190:41913 was elected leader, sending a full tablet report...
W20260811 18:43:47.873610 24381 tablet.cc:2369] T 2d1d5f0630a14484859a4e4e23a44bb7 P 49a0917360fb4c46b936d5d24bc72d08: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:43:47.886845 24147 tablet.cc:2369] T 2d1d5f0630a14484859a4e4e23a44bb7 P 17a2f7dde30b4f2f85a67ca15c4fa88a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:43:47.957064 24425 raft_consensus.cc:482] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:43:47.957512 24425 raft_consensus.cc:504] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a0917360fb4c46b936d5d24bc72d08" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42663 } } peers { permanent_uuid: "cb905b4ee7cf4a7591df70cf1eabba3f" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 42885 } } peers { permanent_uuid: "17a2f7dde30b4f2f85a67ca15c4fa88a" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 34473 } }
I20260811 18:43:47.959753 24425 leader_election.cc:288] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49a0917360fb4c46b936d5d24bc72d08 (127.30.185.131:42663), 17a2f7dde30b4f2f85a67ca15c4fa88a (127.30.185.130:34473)
I20260811 18:43:47.967891 24410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:47.960553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:49633 (local address 127.30.185.131:42663)
0811 18:43:47.960754 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:47.960760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:47.960912 (+ 152us) server_negotiation.cc:407] Connection header received
0811 18:43:47.961261 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:47.961269 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:47.961370 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:47.961447 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:43:47.962313 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.963222 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.965657 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.966862 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.967042 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:47.967120 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:47.967480 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 18:43:47.967616 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 18:43:47.968021 24443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:47.960050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:42663 (local address 127.30.185.129:49633)
0811 18:43:47.960796 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:47.960817 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:43:47.961045 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:43:47.961686 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:43:47.961692 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:43:47.961726 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:43:47.962108 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:47.962144 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.963368 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.963371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:47.965482 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:47.965497 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.967095 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.967102 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:47.967302 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:47.967332 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 18:43:47.967449 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 18:43:47.967596 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"mutex_wait_us":1,"spinlock_wait_cycles":29440,"thread_start_us":701,"threads_started":1}
I20260811 18:43:47.968446 24412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:47.960252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:42641 (local address 127.30.185.130:34473)
0811 18:43:47.960454 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:47.960461 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:47.961805 (+ 1344us) server_negotiation.cc:407] Connection header received
0811 18:43:47.961961 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:47.961967 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:47.962045 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:47.962158 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:43:47.962785 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.963881 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.966099 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.966910 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.967101 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:47.967181 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:47.967742 (+ 561us) server_negotiation.cc:299] Negotiation successful
0811 18:43:47.967892 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 18:43:47.968818 24445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:47.960050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:34473 (local address 127.30.185.129:42641)
0811 18:43:47.961684 (+ 1634us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:47.961700 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:43:47.961819 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:43:47.962206 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:43:47.962211 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:43:47.962239 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:43:47.962604 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:47.962615 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.964052 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.964060 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:47.965943 (+ 1883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:47.965954 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:47.967351 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:47.967358 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:47.967545 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:47.967579 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 18:43:47.968350 (+ 771us) client_negotiation.cc:240] Negotiation successful
0811 18:43:47.968483 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1496,"mutex_wait_us":437,"thread_start_us":843,"threads_started":1}
I20260811 18:43:47.969233 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d1d5f0630a14484859a4e4e23a44bb7" candidate_uuid: "cb905b4ee7cf4a7591df70cf1eabba3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49a0917360fb4c46b936d5d24bc72d08" is_pre_election: true
I20260811 18:43:47.969798 24300 raft_consensus.cc:2454] T 2d1d5f0630a14484859a4e4e23a44bb7 P 49a0917360fb4c46b936d5d24bc72d08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb905b4ee7cf4a7591df70cf1eabba3f in term 0.
I20260811 18:43:47.970001 24064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d1d5f0630a14484859a4e4e23a44bb7" candidate_uuid: "cb905b4ee7cf4a7591df70cf1eabba3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17a2f7dde30b4f2f85a67ca15c4fa88a" is_pre_election: true
I20260811 18:43:47.970526 24064 raft_consensus.cc:2454] T 2d1d5f0630a14484859a4e4e23a44bb7 P 17a2f7dde30b4f2f85a67ca15c4fa88a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb905b4ee7cf4a7591df70cf1eabba3f in term 0.
I20260811 18:43:47.970541 23680 leader_election.cc:302] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49a0917360fb4c46b936d5d24bc72d08, cb905b4ee7cf4a7591df70cf1eabba3f; no voters:
I20260811 18:43:47.970960 24425 raft_consensus.cc:2781] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:43:47.971159 24425 raft_consensus.cc:482] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:43:47.971282 24425 raft_consensus.cc:3037] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:43:47.973798 24425 raft_consensus.cc:504] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a0917360fb4c46b936d5d24bc72d08" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42663 } } peers { permanent_uuid: "cb905b4ee7cf4a7591df70cf1eabba3f" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 42885 } } peers { permanent_uuid: "17a2f7dde30b4f2f85a67ca15c4fa88a" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 34473 } }
I20260811 18:43:47.974519 24425 leader_election.cc:288] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f [CANDIDATE]: Term 1 election: Requested vote from peers 49a0917360fb4c46b936d5d24bc72d08 (127.30.185.131:42663), 17a2f7dde30b4f2f85a67ca15c4fa88a (127.30.185.130:34473)
I20260811 18:43:47.975035 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d1d5f0630a14484859a4e4e23a44bb7" candidate_uuid: "cb905b4ee7cf4a7591df70cf1eabba3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49a0917360fb4c46b936d5d24bc72d08"
I20260811 18:43:47.975093 24064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d1d5f0630a14484859a4e4e23a44bb7" candidate_uuid: "cb905b4ee7cf4a7591df70cf1eabba3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17a2f7dde30b4f2f85a67ca15c4fa88a"
I20260811 18:43:47.975287 24064 raft_consensus.cc:3037] T 2d1d5f0630a14484859a4e4e23a44bb7 P 17a2f7dde30b4f2f85a67ca15c4fa88a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:43:47.975301 24300 raft_consensus.cc:3037] T 2d1d5f0630a14484859a4e4e23a44bb7 P 49a0917360fb4c46b936d5d24bc72d08 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:43:47.978662 24300 raft_consensus.cc:2454] T 2d1d5f0630a14484859a4e4e23a44bb7 P 49a0917360fb4c46b936d5d24bc72d08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb905b4ee7cf4a7591df70cf1eabba3f in term 1.
I20260811 18:43:47.978669 24064 raft_consensus.cc:2454] T 2d1d5f0630a14484859a4e4e23a44bb7 P 17a2f7dde30b4f2f85a67ca15c4fa88a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb905b4ee7cf4a7591df70cf1eabba3f in term 1.
I20260811 18:43:47.979182 23684 leader_election.cc:302] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17a2f7dde30b4f2f85a67ca15c4fa88a, cb905b4ee7cf4a7591df70cf1eabba3f; no voters:
I20260811 18:43:47.979554 24425 raft_consensus.cc:2781] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:43:47.980145 24425 raft_consensus.cc:686] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f [term 1 LEADER]: Becoming Leader. State: Replica: cb905b4ee7cf4a7591df70cf1eabba3f, State: Running, Role: LEADER
I20260811 18:43:47.980738 24425 consensus_queue.cc:237] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a0917360fb4c46b936d5d24bc72d08" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42663 } } peers { permanent_uuid: "cb905b4ee7cf4a7591df70cf1eabba3f" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 42885 } } peers { permanent_uuid: "17a2f7dde30b4f2f85a67ca15c4fa88a" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 34473 } }
I20260811 18:43:47.986142 23532 catalog_manager.cc:5495] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f reported cstate change: term changed from 0 to 1, leader changed from <none> to cb905b4ee7cf4a7591df70cf1eabba3f (127.30.185.129). New cstate: current_term: 1 leader_uuid: "cb905b4ee7cf4a7591df70cf1eabba3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a0917360fb4c46b936d5d24bc72d08" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb905b4ee7cf4a7591df70cf1eabba3f" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 42885 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17a2f7dde30b4f2f85a67ca15c4fa88a" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 34473 } health_report { overall_health: UNKNOWN } } }
I20260811 18:43:48.052052 24409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:48.046557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60840 (local address 127.30.185.129:42885)
0811 18:43:48.046794 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:48.046801 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:48.046822 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:43:48.046972 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:48.046983 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:48.047063 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:48.047225 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:43:48.047634 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:48.048408 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:48.049907 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:48.050321 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:48.050456 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:48.050946 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:43:48.051453 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:43:48.051624 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:48.051676 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 18:43:48.051761 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 18:43:48.061956 24064 raft_consensus.cc:1261] T 2d1d5f0630a14484859a4e4e23a44bb7 P 17a2f7dde30b4f2f85a67ca15c4fa88a [term 1 FOLLOWER]: Refusing update from remote peer cb905b4ee7cf4a7591df70cf1eabba3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:43:48.061980 24300 raft_consensus.cc:1261] T 2d1d5f0630a14484859a4e4e23a44bb7 P 49a0917360fb4c46b936d5d24bc72d08 [term 1 FOLLOWER]: Refusing update from remote peer cb905b4ee7cf4a7591df70cf1eabba3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:43:48.062963 24425 consensus_queue.cc:1035] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "49a0917360fb4c46b936d5d24bc72d08" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:43:48.063304 24447 consensus_queue.cc:1035] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "17a2f7dde30b4f2f85a67ca15c4fa88a" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 34473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:43:48.078042 24457 mvcc.cc:205] Tried to move back new op lower bound from 7317396799730294784 to 7317396799414484992. Current Snapshot: MvccSnapshot[applied={T|T < 7317396799730294784}]
I20260811 18:43:48.082413 24452 mvcc.cc:205] Tried to move back new op lower bound from 7317396799730294784 to 7317396799414484992. Current Snapshot: MvccSnapshot[applied={T|T < 7317396799730294784}]
I20260811 18:43:48.099705 24458 mvcc.cc:205] Tried to move back new op lower bound from 7317396799730294784 to 7317396799414484992. Current Snapshot: MvccSnapshot[applied={T|T < 7317396799730294784}]
I20260811 18:43:48.125025 23532 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:43:48.125463 23532 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:43:48.129642 23532 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b04b3907879f447ab707149cb088f8b7: Sending DeleteTablet for 3 replicas of tablet 2d1d5f0630a14484859a4e4e23a44bb7
I20260811 18:43:48.130890 23764 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d1d5f0630a14484859a4e4e23a44bb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:43:48 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:60836
I20260811 18:43:48.130990 24264 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d1d5f0630a14484859a4e4e23a44bb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:43:48 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:56820
I20260811 18:43:48.131747 24026 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d1d5f0630a14484859a4e4e23a44bb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:43:48 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:53660
I20260811 18:43:48.138520 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 23601
I20260811 18:43:48.149531 24469 tablet_replica.cc:330] T 2d1d5f0630a14484859a4e4e23a44bb7 P 49a0917360fb4c46b936d5d24bc72d08: stopping tablet replica
I20260811 18:43:48.150056 24469 raft_consensus.cc:2229] T 2d1d5f0630a14484859a4e4e23a44bb7 P 49a0917360fb4c46b936d5d24bc72d08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:43:48.150599 24469 raft_consensus.cc:2258] T 2d1d5f0630a14484859a4e4e23a44bb7 P 49a0917360fb4c46b936d5d24bc72d08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:43:48.151425 24470 tablet_replica.cc:330] T 2d1d5f0630a14484859a4e4e23a44bb7 P 17a2f7dde30b4f2f85a67ca15c4fa88a: stopping tablet replica
I20260811 18:43:48.152070 24470 raft_consensus.cc:2229] T 2d1d5f0630a14484859a4e4e23a44bb7 P 17a2f7dde30b4f2f85a67ca15c4fa88a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:43:48.152594 24470 raft_consensus.cc:2258] T 2d1d5f0630a14484859a4e4e23a44bb7 P 17a2f7dde30b4f2f85a67ca15c4fa88a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:43:48.155061 24470 ts_tablet_manager.cc:1905] T 2d1d5f0630a14484859a4e4e23a44bb7 P 17a2f7dde30b4f2f85a67ca15c4fa88a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:43:48.156221 24468 tablet_replica.cc:330] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f: stopping tablet replica
I20260811 18:43:48.156874 24468 raft_consensus.cc:2229] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:43:48.159564 24468 raft_consensus.cc:2258] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:43:48.162191 24468 ts_tablet_manager.cc:1905] T 2d1d5f0630a14484859a4e4e23a44bb7 P cb905b4ee7cf4a7591df70cf1eabba3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:43:48.164701 24409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:48.140453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60856 (local address 127.30.185.129:42885)
0811 18:43:48.140656 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:48.140663 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:48.144247 (+ 3584us) server_negotiation.cc:407] Connection header received
0811 18:43:48.145245 (+ 998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:48.145255 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:48.145359 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:48.145447 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:48.147427 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:48.149072 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:48.151111 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:48.151824 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:48.151980 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:48.156121 (+ 4141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:48.156137 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:48.156155 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:48.156226 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:48.160817 (+ 4591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:48.161881 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:48.161889 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:48.161895 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:48.162060 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:48.162554 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:48.162559 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:48.162561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:48.163071 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:48.163357 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:48.164015 (+ 658us) server_negotiation.cc:299] Negotiation successful
0811 18:43:48.164280 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 18:43:48.166317 23877 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:43:48.166716 24469 ts_tablet_manager.cc:1905] T 2d1d5f0630a14484859a4e4e23a44bb7 P 49a0917360fb4c46b936d5d24bc72d08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:43:48.172325 24470 ts_tablet_manager.cc:1918] T 2d1d5f0630a14484859a4e4e23a44bb7 P 17a2f7dde30b4f2f85a67ca15c4fa88a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:43:48.172523 24470 log.cc:1196] T 2d1d5f0630a14484859a4e4e23a44bb7 P 17a2f7dde30b4f2f85a67ca15c4fa88a: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/2d1d5f0630a14484859a4e4e23a44bb7
I20260811 18:43:48.173045 24470 ts_tablet_manager.cc:1939] T 2d1d5f0630a14484859a4e4e23a44bb7 P 17a2f7dde30b4f2f85a67ca15c4fa88a: Deleting consensus metadata
I20260811 18:43:48.174647 24469 ts_tablet_manager.cc:1918] T 2d1d5f0630a14484859a4e4e23a44bb7 P 49a0917360fb4c46b936d5d24bc72d08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:43:48.174831 24469 log.cc:1196] T 2d1d5f0630a14484859a4e4e23a44bb7 P 49a0917360fb4c46b936d5d24bc72d08: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/2d1d5f0630a14484859a4e4e23a44bb7
I20260811 18:43:48.175199 23512 catalog_manager.cc:4841] TS 17a2f7dde30b4f2f85a67ca15c4fa88a (127.30.185.130:34473): tablet 2d1d5f0630a14484859a4e4e23a44bb7 (table test-table [id=e990ed07d91c43ce956347263ea542e0]) successfully deleted
I20260811 18:43:48.175439 24469 ts_tablet_manager.cc:1939] T 2d1d5f0630a14484859a4e4e23a44bb7 P 49a0917360fb4c46b936d5d24bc72d08: Deleting consensus metadata
I20260811 18:43:48.177414 23507 catalog_manager.cc:4841] TS 49a0917360fb4c46b936d5d24bc72d08 (127.30.185.131:42663): tablet 2d1d5f0630a14484859a4e4e23a44bb7 (table test-table [id=e990ed07d91c43ce956347263ea542e0]) successfully deleted
I20260811 18:43:48.271859 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 23601
I20260811 18:43:48.303043 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 23892
W20260811 18:43:48.302963 23512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.185.129:42885 (error 108)
I20260811 18:43:48.305619 24477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:48.304582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:42885 (local address 127.0.0.1:60872)
0811 18:43:48.305219 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:48.305361 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:42885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":342,"threads_started":1}
W20260811 18:43:48.305974 23512 catalog_manager.cc:4584] TS cb905b4ee7cf4a7591df70cf1eabba3f (127.30.185.129:42885): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2d1d5f0630a14484859a4e4e23a44bb7: Network error: Client connection negotiation failed: client connection to 127.30.185.129:42885: connect: Connection refused (error 111)
I20260811 18:43:48.319224 24412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:48.304247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53664 (local address 127.30.185.130:34473)
0811 18:43:48.304422 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:48.304427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:48.304443 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:43:48.304498 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:48.304502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:48.304560 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:48.304682 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:48.306577 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:48.307409 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:48.308732 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:48.309133 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:48.309317 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:48.311916 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:48.311931 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:48.311947 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:48.311989 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:48.315932 (+ 3943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:48.316760 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:48.316770 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:48.316774 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:48.316961 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:48.317497 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:48.317501 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:48.317503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:48.317907 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:48.318220 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:48.318568 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 18:43:48.318843 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 18:43:48.319972 24135 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:43:48.359756 24477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:48.359199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:42885 (local address 127.0.0.1:60888)
0811 18:43:48.359440 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:48.359574 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:42885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 18:43:48.361044 24477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:48.360611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:42885 (local address 127.0.0.1:60900)
0811 18:43:48.360788 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:48.360876 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:42885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 18:43:48.413373 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 23892
I20260811 18:43:48.422288 24477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:48.421527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:42885 (local address 127.0.0.1:60910)
0811 18:43:48.421920 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:48.422045 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:42885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200}
I20260811 18:43:48.428087 24477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:48.427493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:42885 (local address 127.0.0.1:60922)
0811 18:43:48.427730 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:48.427850 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:42885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 18:43:48.444917 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 24154
I20260811 18:43:48.461654 24410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:48.445921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56836 (local address 127.30.185.131:42663)
0811 18:43:48.446162 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:48.446169 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:48.446195 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:43:48.446352 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:48.446357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:48.446430 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:48.446579 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:48.448098 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:48.448932 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:48.450536 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:48.451015 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:48.451185 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:48.454449 (+ 3264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:48.454467 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:48.454511 (+ 44us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:48.454554 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:48.458391 (+ 3837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:48.459321 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:48.459332 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:48.459335 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:48.459511 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:48.459923 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:48.459929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:48.459932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:48.460352 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:48.460692 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:48.461014 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 18:43:48.461286 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 18:43:48.462409 24372 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:43:48.509488 24477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:48.508853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:42885 (local address 127.0.0.1:60926)
0811 18:43:48.509132 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:48.509291 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:42885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 18:43:48.510792 24477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:48.510250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:42885 (local address 127.0.0.1:60930)
0811 18:43:48.510499 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:48.510610 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:42885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
I20260811 18:43:48.555236 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 24154
I20260811 18:43:48.582561 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 23459
I20260811 18:43:48.583870 23589 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:43:48.704603 24477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:48.703649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:42885 (local address 127.0.0.1:60934)
0811 18:43:48.704185 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:48.704361 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:42885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383}
I20260811 18:43:48.704640 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 23459
I20260811 18:43:48.737706 31462 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 18:43:49.083891 31462 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 18:43:49.084138 31462 mini_ranger.cc:66] Stopping Ranger...
I20260811 18:43:49.440572 31462 mini_ranger.cc:68] Stopped Ranger
2026-08-11 18:43:49.440 UTC [21706] LOG: received smart shutdown request
2026-08-11 18:43:49.444 UTC [21706] LOG: background worker "logical replication launcher" (PID 21724) exited with exit code 1
2026-08-11 18:43:49.445 UTC [21716] LOG: shutting down
2026-08-11 18:43:49.620 UTC [21706] LOG: database system is shut down
2026-08-11T18:43:49Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (56981 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:43:49 dist-test-slave-z8x9 krb5kdc[24495](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:43:49 dist-test-slave-z8x9 krb5kdc[24495](info): set up 2 sockets
Aug 11 18:43:49 dist-test-slave-z8x9 krb5kdc[24495](info): commencing operation
krb5kdc: starting...
W20260811 18:43:51.160405 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.362s user 0.003s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:43:51 dist-test-slave-z8x9 krb5kdc[24495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:43:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:43:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:43:51.474035 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:40847
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:40847,127.30.185.189:38039,127.30.185.188:42005
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:43:51.827001 24516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:43:51.827303 24516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:43:51.827467 24516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:43:51.834724 24516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:43:51.834834 24516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:43:51.834918 24516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:43:51.835014 24516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:43:51.835103 24516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:43:51.845134 24516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:40847,127.30.185.189:38039,127.30.185.188:42005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:40847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:43:51.846575 24516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:43:51.847957 24516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:43:51.862226 24525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:51.862190 24528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:51.862498 24526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:43:51.863184 24516 server_base.cc:1034] running on GCE node
I20260811 18:43:51.864065 24516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:43:51.865636 24516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:43:51.866852 24516 hybrid_clock.cc:648] HybridClock initialized: now 1786473831866804 us; error 49 us; skew 500 ppm
I20260811 18:43:51.867225 24516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:43:51 dist-test-slave-z8x9 krb5kdc[24495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473831, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:43:51.877086 24516 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:43:51.878880 24516 webserver.cc:460] Webserver started at http://127.30.185.190:44783/ using document root <none> and password file <none>
I20260811 18:43:51.879634 24516 fs_manager.cc:359] Metadata directory not provided
I20260811 18:43:51.879776 24516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:43:51.880105 24516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:43:51.883267 24516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "c9fa1c555751445d995292e572929aff"
format_stamp: "Formatted at 2026-08-11 18:43:51 on dist-test-slave-z8x9"
server_key: "fd9d8af1f7683032a2d9f92d831b3242"
server_key_iv: "560645713feb0bd096d7b0fbe44b6895"
server_key_version: "encryptionkey@0"
I20260811 18:43:51.884130 24516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "c9fa1c555751445d995292e572929aff"
format_stamp: "Formatted at 2026-08-11 18:43:51 on dist-test-slave-z8x9"
server_key: "fd9d8af1f7683032a2d9f92d831b3242"
server_key_iv: "560645713feb0bd096d7b0fbe44b6895"
server_key_version: "encryptionkey@0"
I20260811 18:43:51.889461 24516 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 18:43:51.892938 24542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:43:51.894273 24516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:43:51.894475 24516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "c9fa1c555751445d995292e572929aff"
format_stamp: "Formatted at 2026-08-11 18:43:51 on dist-test-slave-z8x9"
server_key: "fd9d8af1f7683032a2d9f92d831b3242"
server_key_iv: "560645713feb0bd096d7b0fbe44b6895"
server_key_version: "encryptionkey@0"
I20260811 18:43:51.894666 24516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:43:51.906468 24516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:43:51.911171 24516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:43:51.911504 24516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:43:51.931460 24516 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:40847
I20260811 18:43:51.931502 24633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:40847 every 8 connection(s)
I20260811 18:43:51.932957 24516 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:43:51.935624 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 24516
I20260811 18:43:51.935887 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:43:51.936720 31462 external_mini_cluster.cc:1428] Setting key d7b7a0dbdd421a1888f3d307a9311868
I20260811 18:43:51.937502 24636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:43:51.947981 24636 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40847 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:43:51 dist-test-slave-z8x9 krb5kdc[24495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473831, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.185.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.189@KRBTEST.COM" created.
I20260811 18:43:51.975023 24643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:51.952009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36686 (local address 127.30.185.190:40847)
0811 18:43:51.953868 (+ 1859us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:51.953880 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:51.953954 (+ 74us) server_negotiation.cc:407] Connection header received
0811 18:43:51.955670 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:51.955681 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:51.956028 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:51.956252 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:51.957609 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:51.959569 (+ 1960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:51.961116 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:51.962206 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:51.962423 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:51.966413 (+ 3990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:51.966428 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:51.966451 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:51.966504 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:51.970221 (+ 3717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:51.971206 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:51.971214 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:51.971216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:51.971390 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:51.971901 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:51.971909 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:51.971915 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:51.972550 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:51.972719 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:51.973572 (+ 853us) server_negotiation.cc:299] Negotiation successful
0811 18:43:51.973860 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":1604,"spinlock_wait_cycles":19712,"thread_start_us":1387,"threads_started":1}
I20260811 18:43:51.975090 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:51.952013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:40847 (local address 127.0.0.1:36686)
0811 18:43:51.953129 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:51.953237 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0811 18:43:51.955253 (+ 2016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:43:51.956282 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:43:51.956287 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:43:51.956891 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:43:51.957388 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:51.957410 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:51.959833 (+ 2423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:51.959838 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:51.960959 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:51.960972 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:51.962436 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:51.962442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:51.962640 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:51.963401 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:43:51.963426 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:43:51.966087 (+ 2661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:43:51.970398 (+ 4311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:51.970412 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:51.970423 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:51.971047 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:51.971545 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:51.971548 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:51.971551 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:51.971774 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:51.972719 (+ 945us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:43:51.972731 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:43:51.973067 (+ 336us) client_negotiation.cc:769] Sending connection context
0811 18:43:51.973399 (+ 332us) client_negotiation.cc:240] Negotiation successful
0811 18:43:51.974079 (+ 680us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":149,"thread_start_us":456,"threads_started":1}
I20260811 18:43:51.978453 24636 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:43:51.979746 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:51.979228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:38039 (local address 127.0.0.1:50212)
0811 18:43:51.979431 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:51.979577 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:38039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 18:43:51.980060 24543 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.185.189:38039: connect: Connection refused (error 111)
I20260811 18:43:51.981863 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:51.981379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:38039 (local address 127.0.0.1:50214)
0811 18:43:51.981567 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:51.981668 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:38039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 18:43:51.982084 24636 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.189:38039: Network error: Client connection negotiation failed: client connection to 127.30.185.189:38039: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.189.keytab.
Entry for principal kudu/127.30.185.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.189.keytab.
I20260811 18:43:52.008273 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.189:38039
--webserver_interface=127.30.185.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.189.keytab
--principal=kudu/127.30.185.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:40847,127.30.185.189:38039,127.30.185.188:42005
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:43:52.031553 24636 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } attempt: 1
I20260811 18:43:52.032899 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.032400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:38039 (local address 127.0.0.1:50226)
0811 18:43:52.032606 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.032742 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:38039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 18:43:52.033919 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.033564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:38039 (local address 127.0.0.1:50234)
0811 18:43:52.033711 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.033787 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:38039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 18:43:52.034101 24636 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.189:38039: Network error: Client connection negotiation failed: client connection to 127.30.185.189:38039: connect: Connection refused (error 111)
I20260811 18:43:52.102339 24636 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } attempt: 2
I20260811 18:43:52.103500 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.103007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:38039 (local address 127.0.0.1:50250)
0811 18:43:52.103259 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.103348 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:38039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 18:43:52.104493 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.104130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:38039 (local address 127.0.0.1:50254)
0811 18:43:52.104294 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.104368 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:38039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 18:43:52.104679 24636 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.189:38039: Network error: Client connection negotiation failed: client connection to 127.30.185.189:38039: connect: Connection refused (error 111)
I20260811 18:43:52.195932 24636 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } attempt: 3
I20260811 18:43:52.197108 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.196647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:38039 (local address 127.0.0.1:50270)
0811 18:43:52.196846 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.196975 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:38039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 18:43:52.198232 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.197881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:38039 (local address 127.0.0.1:50278)
0811 18:43:52.198030 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.198105 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:38039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 18:43:52.198441 24636 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.189:38039: Network error: Client connection negotiation failed: client connection to 127.30.185.189:38039: connect: Connection refused (error 111)
I20260811 18:43:52.341789 24636 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } attempt: 4
I20260811 18:43:52.343119 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.342553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:38039 (local address 127.0.0.1:50280)
0811 18:43:52.342891 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.342982 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:38039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179}
I20260811 18:43:52.344403 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.343950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:38039 (local address 127.0.0.1:50288)
0811 18:43:52.344129 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.344254 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:38039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260811 18:43:52.344589 24636 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.189:38039: Network error: Client connection negotiation failed: client connection to 127.30.185.189:38039: connect: Connection refused (error 111)
W20260811 18:43:52.351580 24647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:43:52.351974 24647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:43:52.352130 24647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:43:52.359760 24647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:43:52.359942 24647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:43:52.360049 24647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:43:52.360159 24647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:43:52.360240 24647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:43:52.370589 24647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:40847,127.30.185.189:38039,127.30.185.188:42005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.189.keytab
--principal=kudu/127.30.185.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.189:38039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.185.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:43:52.372048 24647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:43:52.373484 24647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:43:52.385751 24653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:52.385720 24659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:52.385668 24656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:43:52.386956 24647 server_base.cc:1034] running on GCE node
I20260811 18:43:52.387887 24647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:43:52.389469 24647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:43:52.390750 24647 hybrid_clock.cc:648] HybridClock initialized: now 1786473832390694 us; error 56 us; skew 500 ppm
I20260811 18:43:52.391151 24647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:43:52 dist-test-slave-z8x9 krb5kdc[24495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473832, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:43:52.394881 24647 init.cc:377] Logged in from keytab as kudu/127.30.185.189@KRBTEST.COM (short username kudu)
I20260811 18:43:52.396395 24647 webserver.cc:460] Webserver started at http://127.30.185.189:39045/ using document root <none> and password file <none>
I20260811 18:43:52.397097 24647 fs_manager.cc:359] Metadata directory not provided
I20260811 18:43:52.397291 24647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:43:52.397576 24647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:43:52.400853 24647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-1/data/instance:
uuid: "205cdd3560d246edbbe80782776e13a0"
format_stamp: "Formatted at 2026-08-11 18:43:52 on dist-test-slave-z8x9"
server_key: "eccb90f6862f57753574fa7606fbbb30"
server_key_iv: "709069756520942967933a7392cf0919"
server_key_version: "encryptionkey@0"
I20260811 18:43:52.401713 24647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-1/wal/instance:
uuid: "205cdd3560d246edbbe80782776e13a0"
format_stamp: "Formatted at 2026-08-11 18:43:52 on dist-test-slave-z8x9"
server_key: "eccb90f6862f57753574fa7606fbbb30"
server_key_iv: "709069756520942967933a7392cf0919"
server_key_version: "encryptionkey@0"
I20260811 18:43:52.407109 24647 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:43:52.410845 24667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:43:52.412360 24647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:43:52.412586 24647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-1/wal
uuid: "205cdd3560d246edbbe80782776e13a0"
format_stamp: "Formatted at 2026-08-11 18:43:52 on dist-test-slave-z8x9"
server_key: "eccb90f6862f57753574fa7606fbbb30"
server_key_iv: "709069756520942967933a7392cf0919"
server_key_version: "encryptionkey@0"
I20260811 18:43:52.412837 24647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:43:52.427783 24647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:43:52.432997 24647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:43:52.433375 24647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:43:52.454461 24647 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.189:38039
I20260811 18:43:52.454484 24767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.189:38039 every 8 connection(s)
I20260811 18:43:52.456054 24647 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-1/data/info.pb
I20260811 18:43:52.458432 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 24647
I20260811 18:43:52.458668 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-1/wal/instance
I20260811 18:43:52.459148 31462 external_mini_cluster.cc:1428] Setting key c6e1badcac057d5f1f5ed05c2cd1911a
I20260811 18:43:52.460078 24768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:43:52.467041 24768 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40847 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:43:52 dist-test-slave-z8x9 krb5kdc[24495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473832, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.189@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.185.188@KRBTEST.COM; defaulting to no policy
I20260811 18:43:52.489391 24643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.469852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47460 (local address 127.30.185.190:40847)
0811 18:43:52.470102 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:52.470111 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:52.471351 (+ 1240us) server_negotiation.cc:407] Connection header received
0811 18:43:52.473382 (+ 2031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:52.473388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:52.473488 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:52.473600 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:52.475301 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:52.476000 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:52.477653 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:52.478098 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:52.478242 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:52.482392 (+ 4150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:52.482397 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:52.482400 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:52.482435 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:52.485430 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:52.486495 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:52.486499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:52.486502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:52.486682 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:52.487154 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:52.487157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:52.487159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:52.487456 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:52.487573 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:52.488564 (+ 991us) server_negotiation.cc:299] Negotiation successful
0811 18:43:52.488846 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.188@KRBTEST.COM" created.
I20260811 18:43:52.490260 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.470338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:40847 (local address 127.0.0.1:47460)
0811 18:43:52.471166 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.471208 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:43:52.473039 (+ 1831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:43:52.473850 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:43:52.473860 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:43:52.474528 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:43:52.475123 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:52.475142 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:52.476171 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:52.476175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:52.477502 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:52.477516 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:52.478358 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:52.478365 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:52.478589 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:52.479328 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:43:52.479359 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:43:52.482125 (+ 2766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:43:52.485596 (+ 3471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:52.485608 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:52.485629 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:52.486358 (+ 729us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:52.486832 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:52.486840 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:52.486846 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:52.487047 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:52.487654 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:43:52.487661 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:43:52.488107 (+ 446us) client_negotiation.cc:769] Sending connection context
0811 18:43:52.488400 (+ 293us) client_negotiation.cc:240] Negotiation successful
0811 18:43:52.488827 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":393,"threads_started":1}
I20260811 18:43:52.492116 24768 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:43:52 dist-test-slave-z8x9 krb5kdc[24495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473832, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.189@KRBTEST.COM for kudu/127.30.185.189@KRBTEST.COM
I20260811 18:43:52.510133 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.493033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:38039 (local address 127.0.0.1:48182)
0811 18:43:52.493237 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.493255 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:43:52.493368 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:43:52.494600 (+ 1232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:43:52.494607 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:43:52.495144 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:43:52.495378 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:52.495394 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:52.497248 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:52.497257 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:52.498783 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:52.498802 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:52.499632 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:52.499641 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:52.499890 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:52.500562 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:43:52.500590 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:43:52.503195 (+ 2605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:43:52.507608 (+ 4413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:52.507612 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:52.507615 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:52.507985 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:52.508461 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:52.508464 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:52.508466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:52.508614 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:52.509430 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:43:52.509437 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:43:52.509552 (+ 115us) client_negotiation.cc:769] Sending connection context
0811 18:43:52.509659 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 18:43:52.509863 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 18:43:52.510887 24773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.493313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48182 (local address 127.30.185.189:38039)
0811 18:43:52.494071 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:52.494079 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:52.494111 (+ 32us) server_negotiation.cc:407] Connection header received
0811 18:43:52.494169 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:52.494179 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:52.494419 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:52.494558 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:52.495556 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:52.497063 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:52.498923 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:52.499484 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:52.499580 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:52.503418 (+ 3838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:52.503429 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:52.503443 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:52.503469 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:52.507457 (+ 3988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:52.508136 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:52.508148 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:52.508154 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:52.508342 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:52.508732 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:52.508740 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:52.508746 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:52.509304 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:52.509515 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:52.509892 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 18:43:52.510210 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":410,"threads_started":1}
I20260811 18:43:52.513336 24768 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:43:52.514487 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.514044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:42005 (local address 127.0.0.1:55240)
0811 18:43:52.514208 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.514326 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 18:43:52.514837 24669 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
I20260811 18:43:52.516530 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.516138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:42005 (local address 127.0.0.1:55242)
0811 18:43:52.516335 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.516431 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 18:43:52.516848 24768 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:42005: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.188.keytab.
Entry for principal kudu/127.30.185.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.188.keytab.
I20260811 18:43:52.531962 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.188:42005
--webserver_interface=127.30.185.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.188.keytab
--principal=kudu/127.30.185.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:40847,127.30.185.189:38039,127.30.185.188:42005
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:43:52.566198 24768 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } attempt: 1
I20260811 18:43:52.567427 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.566962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:42005 (local address 127.0.0.1:55246)
0811 18:43:52.567150 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.567257 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 18:43:52.568390 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.568063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:42005 (local address 127.0.0.1:55248)
0811 18:43:52.568221 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.568282 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 18:43:52.568584 24768 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:42005: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
I20260811 18:43:52.636819 24768 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } attempt: 2
I20260811 18:43:52.638118 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.637619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:42005 (local address 127.0.0.1:55264)
0811 18:43:52.637856 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.637950 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 18:43:52.639158 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.638847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:42005 (local address 127.0.0.1:55270)
0811 18:43:52.638974 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.639042 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 18:43:52.639403 24768 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:42005: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
I20260811 18:43:52.643836 24636 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } attempt: 5
Aug 11 18:43:52 dist-test-slave-z8x9 krb5kdc[24495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473831, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for kudu/127.30.185.189@KRBTEST.COM
I20260811 18:43:52.658964 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.644455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:38039 (local address 127.0.0.1:48190)
0811 18:43:52.644676 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.644698 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:43:52.644853 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:43:52.645233 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:43:52.645239 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:43:52.645746 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:43:52.645985 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:52.645998 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:52.646947 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:52.646953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:52.648322 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:52.648338 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:52.649039 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:52.649044 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:52.649273 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:52.649942 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:43:52.649964 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:43:52.652530 (+ 2566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:43:52.656483 (+ 3953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:52.656487 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:52.656490 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:52.656926 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:52.657417 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:52.657420 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:52.657423 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:52.657572 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:52.658195 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:43:52.658198 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:43:52.658339 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 18:43:52.658464 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 18:43:52.658711 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 18:43:52.659178 24773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.644665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48190 (local address 127.30.185.189:38039)
0811 18:43:52.644857 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:52.644863 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:52.644888 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:43:52.644976 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:52.644989 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:52.645066 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:52.645145 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:52.646140 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:52.646809 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:52.648469 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:52.648893 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:52.648981 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:52.652774 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:52.652783 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:52.652791 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:52.652842 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:52.656328 (+ 3486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:52.657093 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:52.657100 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:52.657104 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:52.657300 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:52.657681 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:52.657686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:52.657689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:52.658072 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:52.658229 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:52.658600 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 18:43:52.658845 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 18:43:52.660367 24636 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:43:52.661947 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.661339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:42005 (local address 127.0.0.1:55274)
0811 18:43:52.661663 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.661763 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184}
I20260811 18:43:52.663213 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.662739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:42005 (local address 127.0.0.1:55278)
0811 18:43:52.662958 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.663060 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260811 18:43:52.663450 24636 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:42005: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
I20260811 18:43:52.714715 24636 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } attempt: 1
I20260811 18:43:52.716013 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.715488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:42005 (local address 127.0.0.1:55284)
0811 18:43:52.715717 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.715837 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 18:43:52.717113 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.716710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:42005 (local address 127.0.0.1:55290)
0811 18:43:52.716889 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.716990 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 18:43:52.717391 24636 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:42005: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
I20260811 18:43:52.730631 24768 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } attempt: 3
I20260811 18:43:52.731832 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.731367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:42005 (local address 127.0.0.1:55304)
0811 18:43:52.731558 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.731659 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 18:43:52.732925 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.732504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:42005 (local address 127.0.0.1:55320)
0811 18:43:52.732728 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.732808 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260811 18:43:52.733134 24768 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:42005: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
I20260811 18:43:52.785604 24636 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } attempt: 2
I20260811 18:43:52.786887 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.786390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:42005 (local address 127.0.0.1:55324)
0811 18:43:52.786625 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.786725 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 18:43:52.787923 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.787593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:42005 (local address 127.0.0.1:55332)
0811 18:43:52.787735 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.787812 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 18:43:52.788130 24636 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:42005: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
I20260811 18:43:52.876400 24768 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } attempt: 4
I20260811 18:43:52.877651 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.877104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:42005 (local address 127.0.0.1:55340)
0811 18:43:52.877383 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.877484 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260811 18:43:52.878602 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.878290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:42005 (local address 127.0.0.1:55354)
0811 18:43:52.878433 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.878504 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 18:43:52.878824 24768 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:42005: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
W20260811 18:43:52.891395 24780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:43:52.891752 24780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:43:52.891909 24780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 18:43:52.894347 24636 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } attempt: 3
I20260811 18:43:52.895555 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.895001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:42005 (local address 127.0.0.1:55358)
0811 18:43:52.895239 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.895354 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 18:43:52.896765 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:52.896303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:42005 (local address 127.0.0.1:55374)
0811 18:43:52.896512 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:52.896616 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260811 18:43:52.897017 24636 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:42005: Network error: Client connection negotiation failed: client connection to 127.30.185.188:42005: connect: Connection refused (error 111)
W20260811 18:43:52.900211 24780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:43:52.900328 24780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:43:52.900420 24780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:43:52.900513 24780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:43:52.900592 24780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:43:52.910825 24780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:40847,127.30.185.189:38039,127.30.185.188:42005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.188.keytab
--principal=kudu/127.30.185.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.188:42005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.185.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:43:52.912230 24780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:43:52.913641 24780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:43:52.925763 24791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:52.925725 24787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:52.925812 24788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:43:52.927045 24780 server_base.cc:1034] running on GCE node
I20260811 18:43:52.927888 24780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:43:52.929425 24780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:43:52.930665 24780 hybrid_clock.cc:648] HybridClock initialized: now 1786473832930627 us; error 52 us; skew 500 ppm
I20260811 18:43:52.931082 24780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:43:52 dist-test-slave-z8x9 krb5kdc[24495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473832, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:43:52.934685 24780 init.cc:377] Logged in from keytab as kudu/127.30.185.188@KRBTEST.COM (short username kudu)
I20260811 18:43:52.936264 24780 webserver.cc:460] Webserver started at http://127.30.185.188:45079/ using document root <none> and password file <none>
I20260811 18:43:52.936946 24780 fs_manager.cc:359] Metadata directory not provided
I20260811 18:43:52.937074 24780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:43:52.937448 24780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:43:52.940348 24780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-2/data/instance:
uuid: "afed3bf6af8748ab97966d8c58b3346b"
format_stamp: "Formatted at 2026-08-11 18:43:52 on dist-test-slave-z8x9"
server_key: "6e25797e1d73ffa91dbd40b857815666"
server_key_iv: "b29c9bdf9ac4de50570d46c233f9eda7"
server_key_version: "encryptionkey@0"
I20260811 18:43:52.941202 24780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-2/wal/instance:
uuid: "afed3bf6af8748ab97966d8c58b3346b"
format_stamp: "Formatted at 2026-08-11 18:43:52 on dist-test-slave-z8x9"
server_key: "6e25797e1d73ffa91dbd40b857815666"
server_key_iv: "b29c9bdf9ac4de50570d46c233f9eda7"
server_key_version: "encryptionkey@0"
I20260811 18:43:52.946365 24780 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 18:43:52.949816 24798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:43:52.951165 24780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:43:52.951378 24780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-2/wal
uuid: "afed3bf6af8748ab97966d8c58b3346b"
format_stamp: "Formatted at 2026-08-11 18:43:52 on dist-test-slave-z8x9"
server_key: "6e25797e1d73ffa91dbd40b857815666"
server_key_iv: "b29c9bdf9ac4de50570d46c233f9eda7"
server_key_version: "encryptionkey@0"
I20260811 18:43:52.951596 24780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:43:52.968451 24780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:43:52.972841 24780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:43:52.973152 24780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:43:52.992384 24780 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.188:42005
I20260811 18:43:52.992446 24884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.188:42005 every 8 connection(s)
I20260811 18:43:52.993995 24780 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-2/data/info.pb
I20260811 18:43:52.998020 24885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:43:53.002027 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 24780
I20260811 18:43:53.002247 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786473610573616-31462-0/minicluster-data/master-2/wal/instance
I20260811 18:43:53.002869 31462 external_mini_cluster.cc:1428] Setting key 440f53543759d58337976a927dab7c4c
I20260811 18:43:53.006107 24885 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40847 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:43:53 dist-test-slave-z8x9 krb5kdc[24495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.188@KRBTEST.COM
Aug 11 18:43:53 dist-test-slave-z8x9 krb5kdc[24495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473832, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.188@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:43:53.027767 24893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:53.010304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47468 (local address 127.30.185.190:40847)
0811 18:43:53.010888 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:53.010894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:53.011464 (+ 570us) server_negotiation.cc:407] Connection header received
0811 18:43:53.011650 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:53.011653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:53.011716 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:53.011807 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:53.013123 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.014251 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.016176 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.016864 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.017007 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:53.020684 (+ 3677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:53.020693 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:53.020700 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:53.020742 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:53.024137 (+ 3395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:53.024856 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:53.024863 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:53.024867 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:53.025037 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:53.025628 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:53.025636 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:53.025640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:53.026015 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:53.026127 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:53.027069 (+ 942us) server_negotiation.cc:299] Negotiation successful
0811 18:43:53.027312 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":268,"threads_started":1}
I20260811 18:43:53.028998 24889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:53.004896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55384 (local address 127.30.185.188:42005)
0811 18:43:53.005787 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:53.005799 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:53.005840 (+ 41us) server_negotiation.cc:407] Connection header received
0811 18:43:53.007424 (+ 1584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:53.007446 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:53.007862 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:53.008226 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:53.010163 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.012422 (+ 2259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.014265 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.015330 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.015521 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:53.020426 (+ 4905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:53.020435 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:53.020448 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:53.020481 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:53.024363 (+ 3882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:53.025251 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:53.025260 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:53.025263 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:53.025457 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:53.025916 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:53.025919 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:53.025921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:53.026614 (+ 693us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:53.026934 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:53.027259 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 18:43:53.027605 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":128,"server-negotiator.queue_time_us":682,"thread_start_us":468,"threads_started":1}
I20260811 18:43:53.029018 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:53.010347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:40847 (local address 127.0.0.1:47468)
0811 18:43:53.011305 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:53.011359 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 18:43:53.011531 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:43:53.011892 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:43:53.011898 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:43:53.012708 (+ 810us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:43:53.012971 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:53.012980 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.014427 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.014432 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:53.016004 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:53.016020 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.017145 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.017153 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:53.017330 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:53.018002 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:43:53.018039 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:43:53.020477 (+ 2438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:43:53.024302 (+ 3825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:53.024313 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:53.024323 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:53.024724 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:53.025174 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:53.025204 (+ 30us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:53.025211 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:53.025515 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:53.026144 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:43:53.026153 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:43:53.026617 (+ 464us) client_negotiation.cc:769] Sending connection context
0811 18:43:53.026957 (+ 340us) client_negotiation.cc:240] Negotiation successful
0811 18:43:53.027508 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"mutex_wait_us":49,"spinlock_wait_cycles":22400,"thread_start_us":461,"threads_started":1}
I20260811 18:43:53.030828 24885 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:43:53 dist-test-slave-z8x9 krb5kdc[24495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473832, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.188@KRBTEST.COM for kudu/127.30.185.189@KRBTEST.COM
Aug 11 18:43:53 dist-test-slave-z8x9 krb5kdc[24495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:43:53.046414 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:53.031694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:38039 (local address 127.0.0.1:48206)
0811 18:43:53.031897 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:53.031918 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:43:53.032025 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:43:53.032421 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:43:53.032425 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:43:53.032815 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:43:53.033023 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:53.033039 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.034534 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.034538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:53.035928 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:53.035944 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.036914 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.036917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:53.037069 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:53.037796 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:43:53.037819 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:43:53.040079 (+ 2260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:43:53.043900 (+ 3821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:53.043904 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:53.043926 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:53.044411 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:53.044818 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:53.044822 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:53.044824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:53.044988 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:53.045694 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:43:53.045698 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:43:53.045841 (+ 143us) client_negotiation.cc:769] Sending connection context
0811 18:43:53.045969 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 18:43:53.046169 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 18:43:53.046743 24773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:53.031677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48206 (local address 127.30.185.189:38039)
0811 18:43:53.031928 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:53.031934 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:53.031960 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:43:53.032172 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:53.032176 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:53.032264 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:53.032379 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:53.033202 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.034367 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.036079 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.036771 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.036906 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:53.040292 (+ 3386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:53.040300 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:53.040304 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:53.040372 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:53.043769 (+ 3397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:53.044528 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:53.044535 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:53.044540 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:53.044706 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:53.045104 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:53.045108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:53.045110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:53.045561 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:53.045721 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:53.046102 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 18:43:53.046324 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 18:43:53.047822 24885 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:43:53.050359 24893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:53.032625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47484 (local address 127.30.185.190:40847)
0811 18:43:53.032824 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:53.032830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:53.032848 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:43:53.032900 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:53.032904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:53.032993 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:53.033099 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:53.034733 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.035824 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.037740 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.038390 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.038530 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:53.043260 (+ 4730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:53.043269 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:53.043275 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:53.043320 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:53.047198 (+ 3878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:53.048076 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:53.048082 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:53.048085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:53.048229 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:53.048692 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:53.048697 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:53.048700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:53.049088 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:53.049391 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:53.049770 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 18:43:53.050002 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 18:43:53 dist-test-slave-z8x9 krb5kdc[24495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473832, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.188@KRBTEST.COM for kudu/127.30.185.188@KRBTEST.COM
Aug 11 18:43:53 dist-test-slave-z8x9 krb5kdc[24495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.189@KRBTEST.COM
I20260811 18:43:53.063179 24889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:53.048692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55390 (local address 127.30.185.188:42005)
0811 18:43:53.048906 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:53.048912 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:53.049036 (+ 124us) server_negotiation.cc:407] Connection header received
0811 18:43:53.049224 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:53.049229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:53.049296 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:53.049397 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:53.050229 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.051285 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.052805 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.053297 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.053437 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:53.057141 (+ 3704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:53.057146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:53.057149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:53.057238 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:53.060647 (+ 3409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:53.061282 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:53.061290 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:53.061293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:53.061435 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:53.061846 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:53.061852 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:53.061856 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:53.062118 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:53.062263 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:53.062690 (+ 427us) server_negotiation.cc:299] Negotiation successful
0811 18:43:53.062896 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":67}
I20260811 18:43:53.063570 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:53.048788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:42005 (local address 127.0.0.1:55390)
0811 18:43:53.048947 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:53.048965 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:43:53.049078 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:43:53.049439 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:43:53.049444 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:43:53.049915 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:43:53.050085 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:53.050097 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.051418 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.051425 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:53.052691 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:53.052707 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.053443 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.053448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:53.053728 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:53.054322 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:43:53.054345 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:43:53.056927 (+ 2582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:43:53.060777 (+ 3850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:53.060781 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:53.060783 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:53.061139 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:53.061555 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:53.061560 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:53.061563 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:53.061743 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:53.062243 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:43:53.062247 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:43:53.062436 (+ 189us) client_negotiation.cc:769] Sending connection context
0811 18:43:53.062589 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 18:43:53.063096 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":154}
I20260811 18:43:53.068358 24773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:53.052123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48216 (local address 127.30.185.189:38039)
0811 18:43:53.052287 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:53.052292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:53.052310 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:43:53.052429 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:53.052435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:53.052510 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:53.052620 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:53.053912 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.055028 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.056757 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.057407 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.057507 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:53.062506 (+ 4999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:53.062512 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:53.062518 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:53.062563 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:53.065699 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:53.066394 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:53.066400 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:53.066403 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:53.066573 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:53.066950 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:53.066956 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:53.066959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:53.067221 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:53.067528 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:53.067818 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 18:43:53.068039 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:43:53.068684 24885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b: Bootstrap starting.
I20260811 18:43:53.072712 24885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b: Neither blocks nor log segments found. Creating new log.
I20260811 18:43:53.073593 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:43:53.074074 24885 log.cc:824] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:43:53.074235 24636 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } attempt: 4
I20260811 18:43:53.077414 24885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b: No bootstrap required, opened a new log
Aug 11 18:43:53 dist-test-slave-z8x9 krb5kdc[24495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473831, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for kudu/127.30.185.188@KRBTEST.COM
I20260811 18:43:53.088445 24885 raft_consensus.cc:348] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9fa1c555751445d995292e572929aff" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40847 } } peers { permanent_uuid: "205cdd3560d246edbbe80782776e13a0" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } } peers { permanent_uuid: "afed3bf6af8748ab97966d8c58b3346b" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } }
I20260811 18:43:53.089097 24885 raft_consensus.cc:374] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:43:53.089301 24885 raft_consensus.cc:729] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afed3bf6af8748ab97966d8c58b3346b, State: Initialized, Role: FOLLOWER
I20260811 18:43:53.090224 24885 consensus_queue.cc:260] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9fa1c555751445d995292e572929aff" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40847 } } peers { permanent_uuid: "205cdd3560d246edbbe80782776e13a0" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } } peers { permanent_uuid: "afed3bf6af8748ab97966d8c58b3346b" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } }
I20260811 18:43:53.091979 24911 sys_catalog.cc:455] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9fa1c555751445d995292e572929aff" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40847 } } peers { permanent_uuid: "205cdd3560d246edbbe80782776e13a0" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } } peers { permanent_uuid: "afed3bf6af8748ab97966d8c58b3346b" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } } }
I20260811 18:43:53.092480 24911 sys_catalog.cc:458] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:43:53.093741 24885 sys_catalog.cc:564] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:43:53.095227 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:53.074845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:42005 (local address 127.0.0.1:55406)
0811 18:43:53.075032 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:53.075053 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:43:53.075171 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:43:53.075744 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:43:53.075751 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:43:53.076316 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:43:53.076563 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:53.076578 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.078079 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.078089 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:53.080135 (+ 2046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:53.080156 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.081365 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.081373 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:53.081653 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:53.082452 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:43:53.082478 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:43:53.085653 (+ 3175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:43:53.091180 (+ 5527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:53.091188 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:53.091195 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:53.091868 (+ 673us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:53.092687 (+ 819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:53.092695 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:53.092701 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:53.092934 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:53.093987 (+ 1053us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:43:53.093996 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:43:53.094219 (+ 223us) client_negotiation.cc:769] Sending connection context
0811 18:43:53.094403 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 18:43:53.094809 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 18:43:53.096645 24889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:53.075189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55406 (local address 127.30.185.188:42005)
0811 18:43:53.075399 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:53.075403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:53.075416 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:43:53.075486 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:53.075491 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:53.075563 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:53.075679 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:53.076737 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.077883 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.080334 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.081145 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.081331 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:53.085936 (+ 4605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:53.085946 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:53.085954 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:53.086007 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:53.090683 (+ 4676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:53.092045 (+ 1362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:53.092054 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:53.092060 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:53.092350 (+ 290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:53.093087 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:53.093095 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:53.093101 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:53.093623 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:53.093889 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:53.095771 (+ 1882us) server_negotiation.cc:299] Negotiation successful
0811 18:43:53.096098 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:43:53.106732 24636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff: Bootstrap starting.
I20260811 18:43:53.115051 24636 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff: Neither blocks nor log segments found. Creating new log.
I20260811 18:43:53.117980 24893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:53.083896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47500 (local address 127.30.185.190:40847)
0811 18:43:53.084086 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:53.084092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:53.095388 (+ 11296us) server_negotiation.cc:407] Connection header received
0811 18:43:53.096475 (+ 1087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:53.096484 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:53.096608 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:53.096695 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:53.102192 (+ 5497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.103361 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.105539 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.106183 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.106303 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:53.110272 (+ 3969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:53.110280 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:53.110284 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:53.110338 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:53.114122 (+ 3784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:53.115118 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:53.115123 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:53.115129 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:53.115304 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:53.115817 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:53.115824 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:53.115829 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:53.116242 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:53.116540 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:53.117250 (+ 710us) server_negotiation.cc:299] Negotiation successful
0811 18:43:53.117524 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 18:43:53.122665 24636 log.cc:824] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff: Log is configured to *not* fsync() on all Append() calls
I20260811 18:43:53.128693 24889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:53.084439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55410 (local address 127.30.185.188:42005)
0811 18:43:53.097233 (+ 12794us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:53.097241 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:53.097264 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:43:53.097325 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:53.097332 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:53.097456 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:53.097580 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:53.100605 (+ 3025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.111015 (+ 10410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.113142 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.113978 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.119000 (+ 5022us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:53.119539 (+ 539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:53.119552 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:53.119558 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:53.119602 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:53.123536 (+ 3934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:53.124685 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:53.124694 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:53.124700 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:53.124901 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:53.125503 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:53.125511 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:53.125517 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:53.126021 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:53.127595 (+ 1574us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:53.127777 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 18:43:53.128105 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12607,"thread_start_us":16092,"threads_started":1}
I20260811 18:43:53.133931 24773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:53.093066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48232 (local address 127.30.185.189:38039)
0811 18:43:53.093281 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:53.093287 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:53.103171 (+ 9884us) server_negotiation.cc:407] Connection header received
0811 18:43:53.103825 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:53.103835 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:53.103954 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:53.104045 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:53.107737 (+ 3692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.113515 (+ 5778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.116388 (+ 2873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.117325 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.117481 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:53.124019 (+ 6538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:53.124030 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:53.124039 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:53.124113 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:53.128860 (+ 4747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:53.131153 (+ 2293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:53.131162 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:53.131168 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:53.131411 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:53.131976 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:53.131985 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:53.131991 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:53.132421 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:53.132693 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:53.133233 (+ 540us) server_negotiation.cc:299] Negotiation successful
0811 18:43:53.133492 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 18:43:53.136590 24636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff: No bootstrap required, opened a new log
W20260811 18:43:53.148391 24932 catalog_manager.cc:1558] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:43:53.148634 24932 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:43:53.149657 24636 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9fa1c555751445d995292e572929aff" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40847 } } peers { permanent_uuid: "205cdd3560d246edbbe80782776e13a0" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } } peers { permanent_uuid: "afed3bf6af8748ab97966d8c58b3346b" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } }
I20260811 18:43:53.150075 24636 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:43:53.150197 24636 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9fa1c555751445d995292e572929aff, State: Initialized, Role: FOLLOWER
I20260811 18:43:53.150862 24636 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9fa1c555751445d995292e572929aff" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40847 } } peers { permanent_uuid: "205cdd3560d246edbbe80782776e13a0" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } } peers { permanent_uuid: "afed3bf6af8748ab97966d8c58b3346b" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } }
I20260811 18:43:53.140447 24780 master_runner.cc:186] Error getting master registration for 127.30.185.190:40847: OK, instance_id { permanent_uuid: "c9fa1c555751445d995292e572929aff" instance_seqno: 1786473831925689 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:43:53.152393 24780 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:43:53.151949 24935 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9fa1c555751445d995292e572929aff" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40847 } } peers { permanent_uuid: "205cdd3560d246edbbe80782776e13a0" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } } peers { permanent_uuid: "afed3bf6af8748ab97966d8c58b3346b" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } } }
I20260811 18:43:53.152570 24935 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:43:53.153416 24636 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [sys.catalog]: configured and running, proceeding with master startup.
W20260811 18:43:53.164456 24554 tablet.cc:2369] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:43:53.164803 24954 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:43:53.164934 24954 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:43:53.167090 24516 master_runner.cc:186] Error getting master registration for 127.30.185.189:38039: OK, instance_id { permanent_uuid: "205cdd3560d246edbbe80782776e13a0" instance_seqno: 1786473832448410 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:43:53.174391 24516 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:43:53.178164 24768 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } attempt: 5
Aug 11 18:43:53 dist-test-slave-z8x9 krb5kdc[24495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473832, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.189@KRBTEST.COM for kudu/127.30.185.188@KRBTEST.COM
I20260811 18:43:53.195037 24889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:53.179468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55418 (local address 127.30.185.188:42005)
0811 18:43:53.179646 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:53.179652 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:53.179670 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:43:53.179725 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:53.179731 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:53.179804 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:53.179913 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:53.180741 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.181667 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.182879 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.183373 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.183512 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:53.187270 (+ 3758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:53.187277 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:53.187280 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:53.187315 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:53.190558 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:53.191450 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:53.191455 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:53.191458 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:53.191622 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:53.192197 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:53.192202 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:53.192205 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:53.192614 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:53.192830 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:53.194476 (+ 1646us) server_negotiation.cc:299] Negotiation successful
0811 18:43:53.194740 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:43:53.195720 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:53.178746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:42005 (local address 127.0.0.1:55418)
0811 18:43:53.179104 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 18:43:53.179125 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:43:53.179256 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:43:53.179935 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:43:53.179940 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:43:53.180406 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:43:53.180566 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:53.180575 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.181818 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.181825 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:53.182728 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:43:53.182739 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:43:53.183516 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:43:53.183522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:43:53.183749 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:53.184391 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:43:53.184409 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:43:53.187031 (+ 2622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:43:53.190740 (+ 3709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:53.190749 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:53.190752 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:53.191295 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:53.191770 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:43:53.191777 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:43:53.191781 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:43:53.192003 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:43:53.192896 (+ 893us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:43:53.192901 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:43:53.193090 (+ 189us) client_negotiation.cc:769] Sending connection context
0811 18:43:53.193987 (+ 897us) client_negotiation.cc:240] Negotiation successful
0811 18:43:53.195454 (+ 1467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230}
I20260811 18:43:53.200968 24768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0: Bootstrap starting.
I20260811 18:43:53.205062 24768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0: Neither blocks nor log segments found. Creating new log.
I20260811 18:43:53.206537 24768 log.cc:824] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0: Log is configured to *not* fsync() on all Append() calls
I20260811 18:43:53.209549 24768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0: No bootstrap required, opened a new log
I20260811 18:43:53.217808 24768 raft_consensus.cc:348] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9fa1c555751445d995292e572929aff" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40847 } } peers { permanent_uuid: "205cdd3560d246edbbe80782776e13a0" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } } peers { permanent_uuid: "afed3bf6af8748ab97966d8c58b3346b" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } }
I20260811 18:43:53.218248 24768 raft_consensus.cc:374] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:43:53.218382 24768 raft_consensus.cc:729] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 205cdd3560d246edbbe80782776e13a0, State: Initialized, Role: FOLLOWER
I20260811 18:43:53.219195 24768 consensus_queue.cc:260] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9fa1c555751445d995292e572929aff" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40847 } } peers { permanent_uuid: "205cdd3560d246edbbe80782776e13a0" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } } peers { permanent_uuid: "afed3bf6af8748ab97966d8c58b3346b" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } }
I20260811 18:43:53.220418 24959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9fa1c555751445d995292e572929aff" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40847 } } peers { permanent_uuid: "205cdd3560d246edbbe80782776e13a0" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } } peers { permanent_uuid: "afed3bf6af8748ab97966d8c58b3346b" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } } }
I20260811 18:43:53.220875 24959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:43:53.221782 24768 sys_catalog.cc:564] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 18:43:53.225435 24810 tablet.cc:2369] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:43:53.238691 24978 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:43:53.238974 24978 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:43:53.240342 24647 master_runner.cc:263] No leader master found from master 205cdd3560d246edbbe80782776e13a0
I20260811 18:43:53.240507 24647 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:43:53.333258 24935 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:43:53.333616 24935 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9fa1c555751445d995292e572929aff" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40847 } } peers { permanent_uuid: "205cdd3560d246edbbe80782776e13a0" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } } peers { permanent_uuid: "afed3bf6af8748ab97966d8c58b3346b" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } }
I20260811 18:43:53.335371 24935 leader_election.cc:288] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 205cdd3560d246edbbe80782776e13a0 (127.30.185.189:38039), afed3bf6af8748ab97966d8c58b3346b (127.30.185.188:42005)
I20260811 18:43:53.336136 24716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c9fa1c555751445d995292e572929aff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "205cdd3560d246edbbe80782776e13a0" is_pre_election: true
I20260811 18:43:53.336822 24716 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9fa1c555751445d995292e572929aff in term 0.
I20260811 18:43:53.337881 24543 leader_election.cc:302] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 205cdd3560d246edbbe80782776e13a0, c9fa1c555751445d995292e572929aff; no voters:
I20260811 18:43:53.338516 24935 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:43:53.338703 24935 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:43:53.338815 24935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:43:53.342365 24935 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9fa1c555751445d995292e572929aff" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40847 } } peers { permanent_uuid: "205cdd3560d246edbbe80782776e13a0" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } } peers { permanent_uuid: "afed3bf6af8748ab97966d8c58b3346b" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } }
I20260811 18:43:53.343660 24716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c9fa1c555751445d995292e572929aff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "205cdd3560d246edbbe80782776e13a0"
I20260811 18:43:53.343890 24716 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:43:53.347427 24716 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9fa1c555751445d995292e572929aff in term 1.
I20260811 18:43:53.347999 24543 leader_election.cc:302] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 205cdd3560d246edbbe80782776e13a0, c9fa1c555751445d995292e572929aff; no voters:
I20260811 18:43:53.349109 24839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c9fa1c555751445d995292e572929aff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afed3bf6af8748ab97966d8c58b3346b" is_pre_election: true
I20260811 18:43:53.349776 24839 raft_consensus.cc:2454] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9fa1c555751445d995292e572929aff in term 0.
I20260811 18:43:53.356709 24839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c9fa1c555751445d995292e572929aff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afed3bf6af8748ab97966d8c58b3346b"
I20260811 18:43:53.356974 24839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:43:53.360653 24839 raft_consensus.cc:2454] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9fa1c555751445d995292e572929aff in term 1.
I20260811 18:43:53.361125 24935 leader_election.cc:288] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [CANDIDATE]: Term 1 election: Requested vote from peers 205cdd3560d246edbbe80782776e13a0 (127.30.185.189:38039), afed3bf6af8748ab97966d8c58b3346b (127.30.185.188:42005)
I20260811 18:43:53.361619 24935 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:43:53.362071 24935 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [term 1 LEADER]: Becoming Leader. State: Replica: c9fa1c555751445d995292e572929aff, State: Running, Role: LEADER
I20260811 18:43:53.362912 24935 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9fa1c555751445d995292e572929aff" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40847 } } peers { permanent_uuid: "205cdd3560d246edbbe80782776e13a0" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } } peers { permanent_uuid: "afed3bf6af8748ab97966d8c58b3346b" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } }
I20260811 18:43:53.384187 24981 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9fa1c555751445d995292e572929aff. Latest consensus state: current_term: 1 leader_uuid: "c9fa1c555751445d995292e572929aff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9fa1c555751445d995292e572929aff" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40847 } } peers { permanent_uuid: "205cdd3560d246edbbe80782776e13a0" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } } peers { permanent_uuid: "afed3bf6af8748ab97966d8c58b3346b" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } } }
I20260811 18:43:53.384740 24981 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [sys.catalog]: This master's current role is: LEADER
I20260811 18:43:53.391217 24987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:43:53.393985 24987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:43:53.415653 24839 raft_consensus.cc:1261] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b [term 1 FOLLOWER]: Refusing update from remote peer c9fa1c555751445d995292e572929aff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:43:53.417021 24981 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [LEADER]: Connected to new peer: Peer: permanent_uuid: "afed3bf6af8748ab97966d8c58b3346b" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:43:53.425390 24716 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0 [term 1 FOLLOWER]: Refusing update from remote peer c9fa1c555751445d995292e572929aff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:43:53.426618 24981 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [LEADER]: Connected to new peer: Peer: permanent_uuid: "205cdd3560d246edbbe80782776e13a0" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:43:53.436365 24673 tablet.cc:2369] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:43:53.455578 24959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9fa1c555751445d995292e572929aff. Latest consensus state: current_term: 1 leader_uuid: "c9fa1c555751445d995292e572929aff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9fa1c555751445d995292e572929aff" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40847 } } peers { permanent_uuid: "205cdd3560d246edbbe80782776e13a0" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } } peers { permanent_uuid: "afed3bf6af8748ab97966d8c58b3346b" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } } }
I20260811 18:43:53.455948 24959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:43:53.463250 24990 mvcc.cc:205] Tried to move back new op lower bound from 7317396821656649728 to 7317396821468528640. Current Snapshot: MvccSnapshot[applied={T|T < 7317396821656649728}]
I20260811 18:43:53.469802 24959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c9fa1c555751445d995292e572929aff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9fa1c555751445d995292e572929aff" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40847 } } peers { permanent_uuid: "205cdd3560d246edbbe80782776e13a0" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } } peers { permanent_uuid: "afed3bf6af8748ab97966d8c58b3346b" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } } }
I20260811 18:43:53.470098 24959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:43:53.474213 24935 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c9fa1c555751445d995292e572929aff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9fa1c555751445d995292e572929aff" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40847 } } peers { permanent_uuid: "205cdd3560d246edbbe80782776e13a0" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } } peers { permanent_uuid: "afed3bf6af8748ab97966d8c58b3346b" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } } }
I20260811 18:43:53.474545 24935 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [sys.catalog]: This master's current role is: LEADER
I20260811 18:43:53.476168 24987 catalog_manager.cc:1347] Generated new cluster ID: 830209c080ba4274adadbba6c7f6a635
I20260811 18:43:53.476267 24987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:43:53.496866 24987 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:43:53.501703 24987 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:43:53.507513 24911 sys_catalog.cc:455] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9fa1c555751445d995292e572929aff. Latest consensus state: current_term: 1 leader_uuid: "c9fa1c555751445d995292e572929aff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9fa1c555751445d995292e572929aff" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40847 } } peers { permanent_uuid: "205cdd3560d246edbbe80782776e13a0" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } } peers { permanent_uuid: "afed3bf6af8748ab97966d8c58b3346b" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } } }
I20260811 18:43:53.507863 24911 sys_catalog.cc:458] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:43:53.509220 24935 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c9fa1c555751445d995292e572929aff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9fa1c555751445d995292e572929aff" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40847 } } peers { permanent_uuid: "205cdd3560d246edbbe80782776e13a0" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } } peers { permanent_uuid: "afed3bf6af8748ab97966d8c58b3346b" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } } }
I20260811 18:43:53.509534 24935 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff [sys.catalog]: This master's current role is: LEADER
I20260811 18:43:53.522076 24911 sys_catalog.cc:455] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c9fa1c555751445d995292e572929aff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9fa1c555751445d995292e572929aff" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40847 } } peers { permanent_uuid: "205cdd3560d246edbbe80782776e13a0" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 38039 } } peers { permanent_uuid: "afed3bf6af8748ab97966d8c58b3346b" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 42005 } } }
I20260811 18:43:53.522396 24911 sys_catalog.cc:458] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:43:53.522789 24987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9fa1c555751445d995292e572929aff: Generated new TSK 0
I20260811 18:43:53.525841 24987 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:43:54.150861 24932 catalog_manager.cc:1259] Loaded cluster ID: 830209c080ba4274adadbba6c7f6a635
I20260811 18:43:54.150976 24932 catalog_manager.cc:1552] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b: loading cluster ID for follower catalog manager: success
I20260811 18:43:54.153721 24932 catalog_manager.cc:1574] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b: acquiring CA information for follower catalog manager: success
I20260811 18:43:54.155845 24932 catalog_manager.cc:1602] T 00000000000000000000000000000000 P afed3bf6af8748ab97966d8c58b3346b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:43:54.241102 24978 catalog_manager.cc:1259] Loaded cluster ID: 830209c080ba4274adadbba6c7f6a635
I20260811 18:43:54.241248 24978 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0: loading cluster ID for follower catalog manager: success
I20260811 18:43:54.243692 24978 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0: acquiring CA information for follower catalog manager: success
I20260811 18:43:54.245535 24978 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 205cdd3560d246edbbe80782776e13a0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:43:54.675053 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 24516
I20260811 18:43:54.676661 24623 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:43:54.789093 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 24516
I20260811 18:43:54.825336 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 24647
I20260811 18:43:54.826509 24759 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:43:54.936124 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 24647
I20260811 18:43:54.965350 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 24780
I20260811 18:43:54.966572 24878 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:43:55.075129 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 24780
2026-08-11T18:43:55Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5331 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:43:55 dist-test-slave-z8x9 krb5kdc[25022](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:43:55 dist-test-slave-z8x9 krb5kdc[25022](info): set up 2 sockets
Aug 11 18:43:55 dist-test-slave-z8x9 krb5kdc[25022](info): commencing operation
krb5kdc: starting...
W20260811 18:43:56.498783 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.369s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:43:56 dist-test-slave-z8x9 krb5kdc[25022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:43:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:43:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:43:56.816967 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:43929
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:32793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:43929
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:43:57.176594 25045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:43:57.176910 25045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:43:57.177088 25045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:43:57.184553 25045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:43:57.184679 25045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:43:57.184772 25045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:43:57.184885 25045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:43:57.184949 25045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:43:57.195310 25045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:32793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:43929
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:43929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:43:57.196725 25045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:43:57.198148 25045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:43:57.210937 25056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:57.211081 25057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:57.211052 25063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:43:57.212167 25045 server_base.cc:1034] running on GCE node
I20260811 18:43:57.213078 25045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:43:57.214587 25045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:43:57.215893 25045 hybrid_clock.cc:648] HybridClock initialized: now 1786473837215824 us; error 51 us; skew 500 ppm
I20260811 18:43:57.216301 25045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:43:57 dist-test-slave-z8x9 krb5kdc[25022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473837, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:43:57.223273 25045 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:43:57.224896 25045 webserver.cc:460] Webserver started at http://127.30.185.190:36109/ using document root <none> and password file <none>
I20260811 18:43:57.225704 25045 fs_manager.cc:359] Metadata directory not provided
I20260811 18:43:57.225824 25045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:43:57.226131 25045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:43:57.229485 25045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "557f1ca6c3624ebca8e85d3f047724dc"
format_stamp: "Formatted at 2026-08-11 18:43:57 on dist-test-slave-z8x9"
server_key: "82c2406e9358cae95dffac1cfac5b40e"
server_key_iv: "306cd9b6019fab1cea137f394df975c5"
server_key_version: "encryptionkey@0"
I20260811 18:43:57.230271 25045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "557f1ca6c3624ebca8e85d3f047724dc"
format_stamp: "Formatted at 2026-08-11 18:43:57 on dist-test-slave-z8x9"
server_key: "82c2406e9358cae95dffac1cfac5b40e"
server_key_iv: "306cd9b6019fab1cea137f394df975c5"
server_key_version: "encryptionkey@0"
I20260811 18:43:57.235903 25045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:43:57.239638 25083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:43:57.241241 25045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 18:43:57.241413 25045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "557f1ca6c3624ebca8e85d3f047724dc"
format_stamp: "Formatted at 2026-08-11 18:43:57 on dist-test-slave-z8x9"
server_key: "82c2406e9358cae95dffac1cfac5b40e"
server_key_iv: "306cd9b6019fab1cea137f394df975c5"
server_key_version: "encryptionkey@0"
I20260811 18:43:57.241608 25045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:43:57.255401 25045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:43:57.260224 25045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:43:57.260489 25045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:43:57.280265 25045 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:43929
I20260811 18:43:57.280335 25179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:43929 every 8 connection(s)
I20260811 18:43:57.281806 25045 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:43:57.285740 25181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:43:57.289269 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 25045
I20260811 18:43:57.289528 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:43:57.290441 31462 external_mini_cluster.cc:1428] Setting key a8e86a44b972e0c377d58636d0ef9e24
I20260811 18:43:57.299644 25181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 557f1ca6c3624ebca8e85d3f047724dc: Bootstrap starting.
I20260811 18:43:57.303721 25181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 557f1ca6c3624ebca8e85d3f047724dc: Neither blocks nor log segments found. Creating new log.
I20260811 18:43:57.305157 25181 log.cc:824] T 00000000000000000000000000000000 P 557f1ca6c3624ebca8e85d3f047724dc: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:43:57 dist-test-slave-z8x9 krb5kdc[25022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:43:57.308996 25181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 557f1ca6c3624ebca8e85d3f047724dc: No bootstrap required, opened a new log
I20260811 18:43:57.317953 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:57.292487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33838 (local address 127.30.185.190:43929)
0811 18:43:57.293358 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:57.293371 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:57.293420 (+ 49us) server_negotiation.cc:407] Connection header received
0811 18:43:57.295191 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:57.295215 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:57.295659 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:57.296003 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:57.298029 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:57.299346 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:57.301282 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:57.301976 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:57.302121 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:57.308099 (+ 5978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:57.308113 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:57.308133 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:57.308179 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:57.312654 (+ 4475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:57.313601 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:57.313609 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:57.313619 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:57.313877 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:57.314272 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:57.314277 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:57.314281 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:57.315098 (+ 817us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:57.315458 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:57.315870 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 18:43:57.316399 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":419,"threads_started":1}
I20260811 18:43:57.320042 25181 raft_consensus.cc:348] T 00000000000000000000000000000000 P 557f1ca6c3624ebca8e85d3f047724dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "557f1ca6c3624ebca8e85d3f047724dc" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43929 } }
I20260811 18:43:57.320454 25181 raft_consensus.cc:374] T 00000000000000000000000000000000 P 557f1ca6c3624ebca8e85d3f047724dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:43:57.320566 25181 raft_consensus.cc:729] T 00000000000000000000000000000000 P 557f1ca6c3624ebca8e85d3f047724dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 557f1ca6c3624ebca8e85d3f047724dc, State: Initialized, Role: FOLLOWER
I20260811 18:43:57.321231 25181 consensus_queue.cc:260] T 00000000000000000000000000000000 P 557f1ca6c3624ebca8e85d3f047724dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "557f1ca6c3624ebca8e85d3f047724dc" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43929 } }
I20260811 18:43:57.321529 25181 raft_consensus.cc:388] T 00000000000000000000000000000000 P 557f1ca6c3624ebca8e85d3f047724dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:43:57.321695 25181 raft_consensus.cc:482] T 00000000000000000000000000000000 P 557f1ca6c3624ebca8e85d3f047724dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:43:57.321897 25181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 557f1ca6c3624ebca8e85d3f047724dc [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:43:57.322494 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:43:57.324280 25181 raft_consensus.cc:504] T 00000000000000000000000000000000 P 557f1ca6c3624ebca8e85d3f047724dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "557f1ca6c3624ebca8e85d3f047724dc" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43929 } }
I20260811 18:43:57.324802 25181 leader_election.cc:302] T 00000000000000000000000000000000 P 557f1ca6c3624ebca8e85d3f047724dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 557f1ca6c3624ebca8e85d3f047724dc; no voters:
I20260811 18:43:57.325551 25181 leader_election.cc:288] T 00000000000000000000000000000000 P 557f1ca6c3624ebca8e85d3f047724dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:43:57.325718 25189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 557f1ca6c3624ebca8e85d3f047724dc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:43:57.326782 25189 raft_consensus.cc:686] T 00000000000000000000000000000000 P 557f1ca6c3624ebca8e85d3f047724dc [term 1 LEADER]: Becoming Leader. State: Replica: 557f1ca6c3624ebca8e85d3f047724dc, State: Running, Role: LEADER
I20260811 18:43:57.327298 25189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 557f1ca6c3624ebca8e85d3f047724dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "557f1ca6c3624ebca8e85d3f047724dc" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43929 } }
I20260811 18:43:57.327816 25181 sys_catalog.cc:564] T 00000000000000000000000000000000 P 557f1ca6c3624ebca8e85d3f047724dc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:43:57.330431 25192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 557f1ca6c3624ebca8e85d3f047724dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 557f1ca6c3624ebca8e85d3f047724dc. Latest consensus state: current_term: 1 leader_uuid: "557f1ca6c3624ebca8e85d3f047724dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "557f1ca6c3624ebca8e85d3f047724dc" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43929 } } }
I20260811 18:43:57.330463 25190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 557f1ca6c3624ebca8e85d3f047724dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "557f1ca6c3624ebca8e85d3f047724dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "557f1ca6c3624ebca8e85d3f047724dc" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43929 } } }
I20260811 18:43:57.330943 25190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 557f1ca6c3624ebca8e85d3f047724dc [sys.catalog]: This master's current role is: LEADER
I20260811 18:43:57.331012 25192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 557f1ca6c3624ebca8e85d3f047724dc [sys.catalog]: This master's current role is: LEADER
I20260811 18:43:57.332886 25200 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:43:57.338249 25200 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:43:57.347560 25200 catalog_manager.cc:1347] Generated new cluster ID: 8cdd639aade44be0937874ef94e123c9
I20260811 18:43:57.347713 25200 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:43:57.368513 25200 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:43:57.370500 25200 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:43:57.388281 25200 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 557f1ca6c3624ebca8e85d3f047724dc: Generated new TSK 0
I20260811 18:43:57.389477 25200 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:43:57.435554 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 25045
I20260811 18:43:57.436825 25172 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:43:57.534955 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 25045
2026-08-11T18:43:57Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2453 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:43:57 dist-test-slave-z8x9 krb5kdc[25226](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:43:57 dist-test-slave-z8x9 krb5kdc[25226](info): set up 2 sockets
Aug 11 18:43:57 dist-test-slave-z8x9 krb5kdc[25226](info): commencing operation
krb5kdc: starting...
W20260811 18:43:58.971309 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.388s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:43:59 dist-test-slave-z8x9 krb5kdc[25226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:43:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:43:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:43:59.282828 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:46035
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:39755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:46035
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:43:59.630354 25254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:43:59.630672 25254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:43:59.630842 25254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:43:59.638317 25254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:43:59.638442 25254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:43:59.638557 25254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:43:59.638664 25254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:43:59.638744 25254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:43:59.648752 25254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:39755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:46035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:46035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:43:59.650249 25254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:43:59.651599 25254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:43:59.663820 25260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:59.663884 25263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:43:59.663779 25265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:43:59.664708 25254 server_base.cc:1034] running on GCE node
I20260811 18:43:59.665618 25254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:43:59.667059 25254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:43:59.668294 25254 hybrid_clock.cc:648] HybridClock initialized: now 1786473839668265 us; error 70 us; skew 500 ppm
I20260811 18:43:59.668681 25254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:43:59 dist-test-slave-z8x9 krb5kdc[25226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473839, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:43:59.672003 25254 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:43:59.673625 25254 webserver.cc:460] Webserver started at http://127.30.185.190:43895/ using document root <none> and password file <none>
I20260811 18:43:59.674348 25254 fs_manager.cc:359] Metadata directory not provided
I20260811 18:43:59.674506 25254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:43:59.674830 25254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:43:59.677984 25254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "0d1f7192ff6e4019b62175cfae0a551b"
format_stamp: "Formatted at 2026-08-11 18:43:59 on dist-test-slave-z8x9"
server_key: "bb7590aea56c12beca5a0b2dfaf0c441"
server_key_iv: "317c63af76dcc94e7eca65740af8bb42"
server_key_version: "encryptionkey@0"
I20260811 18:43:59.678784 25254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "0d1f7192ff6e4019b62175cfae0a551b"
format_stamp: "Formatted at 2026-08-11 18:43:59 on dist-test-slave-z8x9"
server_key: "bb7590aea56c12beca5a0b2dfaf0c441"
server_key_iv: "317c63af76dcc94e7eca65740af8bb42"
server_key_version: "encryptionkey@0"
I20260811 18:43:59.684122 25254 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260811 18:43:59.687709 25282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:43:59.689018 25254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:43:59.689302 25254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "0d1f7192ff6e4019b62175cfae0a551b"
format_stamp: "Formatted at 2026-08-11 18:43:59 on dist-test-slave-z8x9"
server_key: "bb7590aea56c12beca5a0b2dfaf0c441"
server_key_iv: "317c63af76dcc94e7eca65740af8bb42"
server_key_version: "encryptionkey@0"
I20260811 18:43:59.689518 25254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:43:59.719094 25254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:43:59.723551 25254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:43:59.723863 25254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:43:59.743357 25254 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:46035
I20260811 18:43:59.743398 25368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:46035 every 8 connection(s)
I20260811 18:43:59.744818 25254 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:43:59.748799 25369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:43:59.752777 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 25254
I20260811 18:43:59.753044 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:43:59.753819 31462 external_mini_cluster.cc:1428] Setting key 915fba848f463894e0702107d0daee6b
I20260811 18:43:59.760088 25369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b: Bootstrap starting.
I20260811 18:43:59.763739 25369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b: Neither blocks nor log segments found. Creating new log.
I20260811 18:43:59.764956 25369 log.cc:824] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:43:59.768478 25369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b: No bootstrap required, opened a new log
Aug 11 18:43:59 dist-test-slave-z8x9 krb5kdc[25226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:43:59.779539 25376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:43:59.755868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55836 (local address 127.30.185.190:46035)
0811 18:43:59.756691 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0811 18:43:59.756706 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:43:59.756750 (+ 44us) server_negotiation.cc:407] Connection header received
0811 18:43:59.758555 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:43:59.758578 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:43:59.758932 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:43:59.759238 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:43:59.761211 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:59.762566 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:59.764525 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:43:59.765314 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:43:59.765442 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:43:59.770554 (+ 5112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:43:59.770565 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:43:59.770584 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:43:59.770627 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:43:59.774747 (+ 4120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:59.775470 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:59.775478 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:59.775487 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:59.775709 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:43:59.776088 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:43:59.776094 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:43:59.776097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:43:59.776786 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:43:59.777213 (+ 427us) server_negotiation.cc:1035] Waiting for connection context
0811 18:43:59.777583 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 18:43:59.778071 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":389,"threads_started":1}
I20260811 18:43:59.779505 25369 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1f7192ff6e4019b62175cfae0a551b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46035 } }
I20260811 18:43:59.780011 25369 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:43:59.780118 25369 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d1f7192ff6e4019b62175cfae0a551b, State: Initialized, Role: FOLLOWER
I20260811 18:43:59.780786 25369 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1f7192ff6e4019b62175cfae0a551b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46035 } }
I20260811 18:43:59.781005 25369 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:43:59.781268 25369 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:43:59.781487 25369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:43:59.783857 25369 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1f7192ff6e4019b62175cfae0a551b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46035 } }
I20260811 18:43:59.784372 25369 leader_election.cc:302] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d1f7192ff6e4019b62175cfae0a551b; no voters:
I20260811 18:43:59.784981 25369 leader_election.cc:288] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:43:59.785157 25382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:43:59.786090 25382 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 1 LEADER]: Becoming Leader. State: Replica: 0d1f7192ff6e4019b62175cfae0a551b, State: Running, Role: LEADER
I20260811 18:43:59.786625 25382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1f7192ff6e4019b62175cfae0a551b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46035 } }
I20260811 18:43:59.787343 25369 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:43:59.789347 25383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d1f7192ff6e4019b62175cfae0a551b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1f7192ff6e4019b62175cfae0a551b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46035 } } }
I20260811 18:43:59.789850 25383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [sys.catalog]: This master's current role is: LEADER
I20260811 18:43:59.792253 25391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:43:59.791678 25386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d1f7192ff6e4019b62175cfae0a551b. Latest consensus state: current_term: 1 leader_uuid: "0d1f7192ff6e4019b62175cfae0a551b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1f7192ff6e4019b62175cfae0a551b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46035 } } }
I20260811 18:43:59.792457 25386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [sys.catalog]: This master's current role is: LEADER
I20260811 18:43:59.798215 25391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:43:59.808640 25391 catalog_manager.cc:1347] Generated new cluster ID: bb74d411909049e09385a3e96c736e18
I20260811 18:43:59.808777 25391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:43:59.828429 25391 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:43:59.830562 25391 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:43:59.844467 25391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b: Generated new TSK 0
I20260811 18:43:59.845822 25391 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.185.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
I20260811 18:43:59.918900 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:46035
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:39755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:44:00.268272 25423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:44:00.268568 25423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:44:00.268667 25423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:44:00.275949 25423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:44:00.276065 25423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:44:00.276187 25423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:44:00.286034 25423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:39755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:46035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:44:00.287392 25423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:44:00.288924 25423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:44:00.303306 25435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:00.303257 25439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:00.303272 25436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:44:00.304615 25423 server_base.cc:1034] running on GCE node
I20260811 18:44:00.305371 25423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:44:00.306368 25423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:44:00.307607 25423 hybrid_clock.cc:648] HybridClock initialized: now 1786473840307559 us; error 44 us; skew 500 ppm
I20260811 18:44:00.307953 25423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:44:00 dist-test-slave-z8x9 krb5kdc[25226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473840, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:44:00.311270 25423 init.cc:377] Logged in from keytab as kudu/127.30.185.129@KRBTEST.COM (short username kudu)
I20260811 18:44:00.312820 25423 webserver.cc:460] Webserver started at http://127.30.185.129:44951/ using document root <none> and password file <none>
I20260811 18:44:00.313632 25423 fs_manager.cc:359] Metadata directory not provided
I20260811 18:44:00.313786 25423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:44:00.314108 25423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:44:00.317292 25423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "a9814a8e125c4eeb8a479d67ef8b2045"
format_stamp: "Formatted at 2026-08-11 18:44:00 on dist-test-slave-z8x9"
server_key: "bcc3bb38a8b883c752bcced5e84fdab6"
server_key_iv: "692178b074c55d6786d25c63b6a9f336"
server_key_version: "encryptionkey@0"
I20260811 18:44:00.318114 25423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "a9814a8e125c4eeb8a479d67ef8b2045"
format_stamp: "Formatted at 2026-08-11 18:44:00 on dist-test-slave-z8x9"
server_key: "bcc3bb38a8b883c752bcced5e84fdab6"
server_key_iv: "692178b074c55d6786d25c63b6a9f336"
server_key_version: "encryptionkey@0"
I20260811 18:44:00.323457 25423 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260811 18:44:00.327283 25455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:00.328778 25423 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:44:00.328953 25423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "a9814a8e125c4eeb8a479d67ef8b2045"
format_stamp: "Formatted at 2026-08-11 18:44:00 on dist-test-slave-z8x9"
server_key: "bcc3bb38a8b883c752bcced5e84fdab6"
server_key_iv: "692178b074c55d6786d25c63b6a9f336"
server_key_version: "encryptionkey@0"
I20260811 18:44:00.329177 25423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:44:00.359841 25423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:44:00.364158 25423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:44:00.364494 25423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:44:00.365585 25423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:44:00.366967 25423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:44:00.367046 25423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:00.367204 25423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:44:00.367285 25423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:00.402446 25423 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:37699
I20260811 18:44:00.402489 25652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:37699 every 8 connection(s)
I20260811 18:44:00.403820 25423 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:44:00.410738 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 25423
I20260811 18:44:00.410996 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:44:00.411899 31462 external_mini_cluster.cc:1428] Setting key 96e991128292a9ed7896e4ffc265f09c
Aug 11 18:44:00 dist-test-slave-z8x9 krb5kdc[25226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473840, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:44:00.426426 25662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:00.406437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:40133 (local address 127.30.185.190:46035)
0811 18:44:00.406916 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:00.406920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:00.407724 (+ 804us) server_negotiation.cc:407] Connection header received
0811 18:44:00.409633 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:00.409638 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:00.409731 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:00.409857 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:00.411950 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:00.412865 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:00.415033 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:00.415435 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:00.415580 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:00.420232 (+ 4652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:00.420238 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:00.420241 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:00.420273 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:00.423042 (+ 2769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:00.424031 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:00.424036 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:00.424038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:00.424158 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:00.424671 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:00.424674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:00.424676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:00.424930 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:00.425041 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:00.425988 (+ 947us) server_negotiation.cc:299] Negotiation successful
0811 18:44:00.426162 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":260,"threads_started":1}
I20260811 18:44:00.427583 25663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:00.406880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.129:40133)
0811 18:44:00.407567 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:00.407607 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:00.409363 (+ 1756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:00.410095 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:00.410104 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:00.410908 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:44:00.411742 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:00.411770 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:00.413051 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:00.413056 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:00.414855 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:00.414875 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:00.415668 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:00.415679 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:00.416040 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:00.416822 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:44:00.416856 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:44:00.419979 (+ 3123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:44:00.423205 (+ 3226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:00.423218 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:00.423238 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:00.423904 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:00.424264 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:00.424271 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:00.424277 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:00.424554 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:00.425055 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:44:00.425066 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:44:00.425576 (+ 510us) client_negotiation.cc:769] Sending connection context
0811 18:44:00.425891 (+ 315us) client_negotiation.cc:240] Negotiation successful
0811 18:44:00.426496 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":278,"threads_started":1}
I20260811 18:44:00.429188 25656 heartbeater.cc:347] Connected to a master server at 127.30.185.190:46035
I20260811 18:44:00.429523 25656 heartbeater.cc:464] Registering TS with master...
I20260811 18:44:00.430281 25656 heartbeater.cc:511] Master 127.30.185.190:46035 requested a full tablet report, sending...
I20260811 18:44:00.432545 25305 ts_manager.cc:194] Registered new tserver with Master: a9814a8e125c4eeb8a479d67ef8b2045 (127.30.185.129:37699)
I20260811 18:44:00.434180 25305 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:40133
WARNING: no policy specified for kudu/127.30.185.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
I20260811 18:44:00.480482 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:46035
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:39755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:44:00.841768 25672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:44:00.842132 25672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:44:00.842319 25672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:44:00.849750 25672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:44:00.849932 25672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:44:00.850134 25672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:44:00.860455 25672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:39755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:46035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:44:00.861927 25672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:44:00.863303 25672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:44:00.877918 25678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:00.878090 25683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:00.877967 25681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:44:00.879127 25672 server_base.cc:1034] running on GCE node
I20260811 18:44:00.879779 25672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:44:00.880750 25672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:44:00.881953 25672 hybrid_clock.cc:648] HybridClock initialized: now 1786473840881904 us; error 38 us; skew 500 ppm
I20260811 18:44:00.882309 25672 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:44:00 dist-test-slave-z8x9 krb5kdc[25226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473840, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:44:00.885624 25672 init.cc:377] Logged in from keytab as kudu/127.30.185.130@KRBTEST.COM (short username kudu)
I20260811 18:44:00.887286 25672 webserver.cc:460] Webserver started at http://127.30.185.130:43679/ using document root <none> and password file <none>
I20260811 18:44:00.888048 25672 fs_manager.cc:359] Metadata directory not provided
I20260811 18:44:00.888206 25672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:44:00.888531 25672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:44:00.892820 25672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "97fc6445d9244db78dd6284e19b22409"
format_stamp: "Formatted at 2026-08-11 18:44:00 on dist-test-slave-z8x9"
server_key: "b74b4025ff34bc4a1f221c5b346f0227"
server_key_iv: "48f52eccb0e58911547dcfa730d986f3"
server_key_version: "encryptionkey@0"
I20260811 18:44:00.893844 25672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "97fc6445d9244db78dd6284e19b22409"
format_stamp: "Formatted at 2026-08-11 18:44:00 on dist-test-slave-z8x9"
server_key: "b74b4025ff34bc4a1f221c5b346f0227"
server_key_iv: "48f52eccb0e58911547dcfa730d986f3"
server_key_version: "encryptionkey@0"
I20260811 18:44:00.899559 25672 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 18:44:00.903195 25694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:00.904599 25672 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 18:44:00.904783 25672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "97fc6445d9244db78dd6284e19b22409"
format_stamp: "Formatted at 2026-08-11 18:44:00 on dist-test-slave-z8x9"
server_key: "b74b4025ff34bc4a1f221c5b346f0227"
server_key_iv: "48f52eccb0e58911547dcfa730d986f3"
server_key_version: "encryptionkey@0"
I20260811 18:44:00.904996 25672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:44:00.929875 25672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:44:00.934297 25672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:44:00.934628 25672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:44:00.935699 25672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:44:00.937160 25672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:44:00.937315 25672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 18:44:00.937513 25672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:44:00.937592 25672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:00.973492 25672 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:33119
I20260811 18:44:00.973536 25871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:33119 every 8 connection(s)
I20260811 18:44:00.974881 25672 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:44:00.978679 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 25672
I20260811 18:44:00.978894 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:44:00.979362 31462 external_mini_cluster.cc:1428] Setting key 9d616a0fd51e9660350836711e45280d
Aug 11 18:44:00 dist-test-slave-z8x9 krb5kdc[25226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473840, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:44:00.998600 25876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:00.977860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:35317 (local address 127.30.185.190:46035)
0811 18:44:00.978383 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:00.978392 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:00.979815 (+ 1423us) server_negotiation.cc:407] Connection header received
0811 18:44:00.982097 (+ 2282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:00.982102 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:00.982189 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:00.982327 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:00.983901 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:00.985092 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:00.986753 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:00.987535 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:00.987685 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:00.991301 (+ 3616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:00.991309 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:00.991317 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:00.991370 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:00.995126 (+ 3756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:00.996142 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:00.996146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:00.996148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:00.996287 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:00.996789 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:00.996792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:00.996794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:00.997047 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:00.997237 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:00.998118 (+ 881us) server_negotiation.cc:299] Negotiation successful
0811 18:44:00.998307 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":256,"threads_started":1}
I20260811 18:44:01.000273 25878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:00.978566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.130:35317)
0811 18:44:00.979613 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:00.979674 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:00.981813 (+ 2139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:00.982566 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:00.982575 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:00.983136 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:44:00.983719 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:00.983737 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:00.985315 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:00.985319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:00.986596 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:00.986609 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:00.987754 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:00.987759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:00.988020 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:00.988771 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:44:00.988800 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:44:00.991064 (+ 2264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:44:00.995271 (+ 4207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:00.995283 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:00.995304 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:00.996009 (+ 705us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:00.996398 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:00.996406 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:00.996413 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:00.996669 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:00.997222 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:44:00.997234 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:44:00.997720 (+ 486us) client_negotiation.cc:769] Sending connection context
0811 18:44:00.998048 (+ 328us) client_negotiation.cc:240] Negotiation successful
0811 18:44:00.998715 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":419,"threads_started":1}
I20260811 18:44:01.001890 25873 heartbeater.cc:347] Connected to a master server at 127.30.185.190:46035
I20260811 18:44:01.002223 25873 heartbeater.cc:464] Registering TS with master...
I20260811 18:44:01.002986 25873 heartbeater.cc:511] Master 127.30.185.190:46035 requested a full tablet report, sending...
I20260811 18:44:01.004541 25305 ts_manager.cc:194] Registered new tserver with Master: 97fc6445d9244db78dd6284e19b22409 (127.30.185.130:33119)
I20260811 18:44:01.005455 25305 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:35317
WARNING: no policy specified for kudu/127.30.185.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
I20260811 18:44:01.051159 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:46035
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:39755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:44:01.392894 25885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:44:01.393409 25885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:44:01.393597 25885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:44:01.401710 25885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:44:01.401887 25885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:44:01.402137 25885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:44:01.414372 25885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:39755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:46035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:44:01.415577 25885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:44:01.416826 25885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:44:01.430301 25893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:01.430286 25895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:01.430441 25897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:44:01.431258 25885 server_base.cc:1034] running on GCE node
I20260811 18:44:01.431891 25885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:44:01.432924 25885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:44:01.434154 25885 hybrid_clock.cc:648] HybridClock initialized: now 1786473841434068 us; error 76 us; skew 500 ppm
I20260811 18:44:01.434494 25885 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:44:01 dist-test-slave-z8x9 krb5kdc[25226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473841, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:44:01.437827 25885 init.cc:377] Logged in from keytab as kudu/127.30.185.131@KRBTEST.COM (short username kudu)
I20260811 18:44:01.437983 25656 heartbeater.cc:503] Master 127.30.185.190:46035 was elected leader, sending a full tablet report...
I20260811 18:44:01.439383 25885 webserver.cc:460] Webserver started at http://127.30.185.131:40431/ using document root <none> and password file <none>
I20260811 18:44:01.440202 25885 fs_manager.cc:359] Metadata directory not provided
I20260811 18:44:01.440356 25885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:44:01.440657 25885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:44:01.443615 25885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "21050d666d634b858289431412d37063"
format_stamp: "Formatted at 2026-08-11 18:44:01 on dist-test-slave-z8x9"
server_key: "99c07aea91736332afe04d955b569409"
server_key_iv: "05e37c42b36a428bd37d3dc2190a60d9"
server_key_version: "encryptionkey@0"
I20260811 18:44:01.444418 25885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "21050d666d634b858289431412d37063"
format_stamp: "Formatted at 2026-08-11 18:44:01 on dist-test-slave-z8x9"
server_key: "99c07aea91736332afe04d955b569409"
server_key_iv: "05e37c42b36a428bd37d3dc2190a60d9"
server_key_version: "encryptionkey@0"
I20260811 18:44:01.449628 25885 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 18:44:01.453261 25914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:01.454558 25885 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:44:01.454751 25885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "21050d666d634b858289431412d37063"
format_stamp: "Formatted at 2026-08-11 18:44:01 on dist-test-slave-z8x9"
server_key: "99c07aea91736332afe04d955b569409"
server_key_iv: "05e37c42b36a428bd37d3dc2190a60d9"
server_key_version: "encryptionkey@0"
I20260811 18:44:01.454963 25885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:44:01.471172 25885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:44:01.475759 25885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:44:01.476060 25885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:44:01.477042 25885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:44:01.478449 25885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:44:01.478560 25885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:01.478716 25885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:44:01.478791 25885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:01.514529 25885 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:34335
I20260811 18:44:01.514566 26109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:34335 every 8 connection(s)
I20260811 18:44:01.515883 25885 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:44:01.521956 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 25885
I20260811 18:44:01.522188 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:44:01.522656 31462 external_mini_cluster.cc:1428] Setting key b3ea50c0bb59491885ca67bf717cbe23
Aug 11 18:44:01 dist-test-slave-z8x9 krb5kdc[25226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473841, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:44:01.536729 26113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:01.518632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:54373 (local address 127.30.185.190:46035)
0811 18:44:01.519064 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:01.519068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:01.520157 (+ 1089us) server_negotiation.cc:407] Connection header received
0811 18:44:01.522129 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:01.522134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:01.522244 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:01.522387 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:01.523969 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.524786 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.526407 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.527013 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.527239 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:01.530779 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:01.530784 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:01.530787 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:01.530829 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:01.533701 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:01.534420 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:01.534424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:01.534426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:01.534546 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:01.534941 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:01.534944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:01.534946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:01.535211 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:01.535330 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:01.536072 (+ 742us) server_negotiation.cc:299] Negotiation successful
0811 18:44:01.536313 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":206,"threads_started":1}
I20260811 18:44:01.537892 26114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:01.519176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.131:54373)
0811 18:44:01.519965 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:01.520015 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:01.521845 (+ 1830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:01.522639 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:01.522650 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:01.523215 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:44:01.523788 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:01.523806 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.524964 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.524967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:01.526261 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:01.526274 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.527278 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.527287 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:01.527563 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:01.528234 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:44:01.528258 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:44:01.530566 (+ 2308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:44:01.533858 (+ 3292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:01.533865 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:01.533877 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:01.534307 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:01.534638 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:01.534642 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:01.534644 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:01.534828 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:01.535321 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:44:01.535327 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:44:01.535654 (+ 327us) client_negotiation.cc:769] Sending connection context
0811 18:44:01.535960 (+ 306us) client_negotiation.cc:240] Negotiation successful
0811 18:44:01.536499 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":314,"threads_started":1}
I20260811 18:44:01.539301 26110 heartbeater.cc:347] Connected to a master server at 127.30.185.190:46035
I20260811 18:44:01.539608 26110 heartbeater.cc:464] Registering TS with master...
I20260811 18:44:01.540334 26110 heartbeater.cc:511] Master 127.30.185.190:46035 requested a full tablet report, sending...
I20260811 18:44:01.541736 25305 ts_manager.cc:194] Registered new tserver with Master: 21050d666d634b858289431412d37063 (127.30.185.131:34335)
I20260811 18:44:01.542568 25305 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:54373
I20260811 18:44:01.550405 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:44:01.567663 26113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:01.555001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55842 (local address 127.30.185.190:46035)
0811 18:44:01.555156 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:01.555161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:01.555681 (+ 520us) server_negotiation.cc:407] Connection header received
0811 18:44:01.555825 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:01.555828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:01.555913 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:01.555997 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:01.557445 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.558172 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.559553 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.560056 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.560151 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:01.562593 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:01.562598 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:01.562601 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:01.562631 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:01.565319 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:01.566002 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:01.566006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:01.566008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:01.566138 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:01.566458 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:01.566461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:01.566463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:01.566781 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:01.566970 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:01.567268 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 18:44:01.567445 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 18:44:01.571791 25305 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:44:01.578634 25305 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:44:01.618564 26144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:01.591400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56452 (local address 127.30.185.129:37699)
0811 18:44:01.606646 (+ 15246us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:01.606655 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:01.606690 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:44:01.606812 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:01.606820 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:01.607108 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:01.607324 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:44:01.609012 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.611310 (+ 2298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.615582 (+ 4272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.617104 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.617369 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:01.617580 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:01.618032 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 18:44:01.618211 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15062,"thread_start_us":15509,"threads_started":1}
I20260811 18:44:01.618646 26146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:01.591039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:37699 (local address 127.0.0.1:56452)
0811 18:44:01.604848 (+ 13809us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:01.604951 (+ 103us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:01.605284 (+ 333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:01.607608 (+ 2324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:01.607615 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:01.607654 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:44:01.608620 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:01.608641 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.611542 (+ 2901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.611552 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:01.615355 (+ 3803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:01.615376 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.617524 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.617530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:01.617763 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:01.617816 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 18:44:01.617962 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 18:44:01.618154 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13362,"mutex_wait_us":58,"thread_start_us":15195,"threads_started":1}
I20260811 18:44:01.620774 26148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:01.591611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45278 (local address 127.30.185.131:34335)
0811 18:44:01.606148 (+ 14537us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:01.606160 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:01.606824 (+ 664us) server_negotiation.cc:407] Connection header received
0811 18:44:01.606932 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:01.606942 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:01.607184 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:01.608400 (+ 1216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:44:01.609286 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.611655 (+ 2369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.616881 (+ 5226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.618383 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.620235 (+ 1852us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:01.620365 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:01.620417 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 18:44:01.620529 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14347,"thread_start_us":14178,"threads_started":1}
I20260811 18:44:01.621309 26141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:01.591447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:34335 (local address 127.0.0.1:45278)
0811 18:44:01.606522 (+ 15075us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:01.606540 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:01.606681 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:01.607483 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:01.607491 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:01.607523 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:44:01.608571 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:01.608598 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.611868 (+ 3270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.611877 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:01.616694 (+ 4817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:01.616708 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.619467 (+ 2759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.619473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:01.619644 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:01.619677 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 18:44:01.619743 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 18:44:01.619893 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14955,"mutex_wait_us":1503,"thread_start_us":14744,"threads_started":1}
I20260811 18:44:01.623212 25545 tablet_service.cc:1467] Processing CreateTablet for tablet 1f1f3a431e644d269c05d2513f9d99ab (DEFAULT_TABLE table=test-table [id=9e7c474f3123454282844fe39c6d31cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:44:01.624879 25545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f1f3a431e644d269c05d2513f9d99ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:44:01.625484 26149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:01.607439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:33119 (local address 127.0.0.1:49462)
0811 18:44:01.615782 (+ 8343us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:01.615809 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:01.615969 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:01.617105 (+ 1136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:01.617110 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:01.617154 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:44:01.617670 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:01.617684 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.619713 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.619717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:01.622533 (+ 2816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:01.622552 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.624689 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.624695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:01.624856 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:01.624891 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 18:44:01.624977 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 18:44:01.625102 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8136,"thread_start_us":7910,"threads_started":1}
I20260811 18:44:01.625296 25996 tablet_service.cc:1467] Processing CreateTablet for tablet 1f1f3a431e644d269c05d2513f9d99ab (DEFAULT_TABLE table=test-table [id=9e7c474f3123454282844fe39c6d31cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:44:01.626370 26150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:01.607492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49462 (local address 127.30.185.130:33119)
0811 18:44:01.616445 (+ 8953us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:01.616455 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:01.616486 (+ 31us) server_negotiation.cc:407] Connection header received
0811 18:44:01.616582 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:01.616591 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:01.616868 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:01.617096 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:44:01.617880 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.619541 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.622749 (+ 3208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.624197 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.624387 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:01.624546 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:01.625840 (+ 1294us) server_negotiation.cc:299] Negotiation successful
0811 18:44:01.626021 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8783,"thread_start_us":8664,"threads_started":1}
I20260811 18:44:01.626754 25996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f1f3a431e644d269c05d2513f9d99ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:44:01.630051 25774 tablet_service.cc:1467] Processing CreateTablet for tablet 1f1f3a431e644d269c05d2513f9d99ab (DEFAULT_TABLE table=test-table [id=9e7c474f3123454282844fe39c6d31cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:44:01.631491 25774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f1f3a431e644d269c05d2513f9d99ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:44:01.637266 26151 tablet_bootstrap.cc:492] T 1f1f3a431e644d269c05d2513f9d99ab P a9814a8e125c4eeb8a479d67ef8b2045: Bootstrap starting.
I20260811 18:44:01.641736 26151 tablet_bootstrap.cc:654] T 1f1f3a431e644d269c05d2513f9d99ab P a9814a8e125c4eeb8a479d67ef8b2045: Neither blocks nor log segments found. Creating new log.
I20260811 18:44:01.642293 26152 tablet_bootstrap.cc:492] T 1f1f3a431e644d269c05d2513f9d99ab P 21050d666d634b858289431412d37063: Bootstrap starting.
I20260811 18:44:01.643280 26151 log.cc:824] T 1f1f3a431e644d269c05d2513f9d99ab P a9814a8e125c4eeb8a479d67ef8b2045: Log is configured to *not* fsync() on all Append() calls
I20260811 18:44:01.646569 26155 tablet_bootstrap.cc:492] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409: Bootstrap starting.
I20260811 18:44:01.646692 26151 tablet_bootstrap.cc:492] T 1f1f3a431e644d269c05d2513f9d99ab P a9814a8e125c4eeb8a479d67ef8b2045: No bootstrap required, opened a new log
I20260811 18:44:01.647047 26152 tablet_bootstrap.cc:654] T 1f1f3a431e644d269c05d2513f9d99ab P 21050d666d634b858289431412d37063: Neither blocks nor log segments found. Creating new log.
I20260811 18:44:01.647379 26151 ts_tablet_manager.cc:1397] T 1f1f3a431e644d269c05d2513f9d99ab P a9814a8e125c4eeb8a479d67ef8b2045: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260811 18:44:01.648447 26152 log.cc:824] T 1f1f3a431e644d269c05d2513f9d99ab P 21050d666d634b858289431412d37063: Log is configured to *not* fsync() on all Append() calls
I20260811 18:44:01.650285 26155 tablet_bootstrap.cc:654] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409: Neither blocks nor log segments found. Creating new log.
I20260811 18:44:01.651557 26155 log.cc:824] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409: Log is configured to *not* fsync() on all Append() calls
I20260811 18:44:01.651872 26152 tablet_bootstrap.cc:492] T 1f1f3a431e644d269c05d2513f9d99ab P 21050d666d634b858289431412d37063: No bootstrap required, opened a new log
I20260811 18:44:01.652168 26152 ts_tablet_manager.cc:1397] T 1f1f3a431e644d269c05d2513f9d99ab P 21050d666d634b858289431412d37063: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260811 18:44:01.654814 26155 tablet_bootstrap.cc:492] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409: No bootstrap required, opened a new log
I20260811 18:44:01.655094 26155 ts_tablet_manager.cc:1397] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 18:44:01.655978 26151 raft_consensus.cc:348] T 1f1f3a431e644d269c05d2513f9d99ab P a9814a8e125c4eeb8a479d67ef8b2045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37699 } } peers { permanent_uuid: "21050d666d634b858289431412d37063" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34335 } } peers { permanent_uuid: "97fc6445d9244db78dd6284e19b22409" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33119 } }
I20260811 18:44:01.656458 26151 raft_consensus.cc:374] T 1f1f3a431e644d269c05d2513f9d99ab P a9814a8e125c4eeb8a479d67ef8b2045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:44:01.656567 26151 raft_consensus.cc:729] T 1f1f3a431e644d269c05d2513f9d99ab P a9814a8e125c4eeb8a479d67ef8b2045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9814a8e125c4eeb8a479d67ef8b2045, State: Initialized, Role: FOLLOWER
I20260811 18:44:01.657261 26151 consensus_queue.cc:260] T 1f1f3a431e644d269c05d2513f9d99ab P a9814a8e125c4eeb8a479d67ef8b2045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37699 } } peers { permanent_uuid: "21050d666d634b858289431412d37063" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34335 } } peers { permanent_uuid: "97fc6445d9244db78dd6284e19b22409" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33119 } }
I20260811 18:44:01.659085 26151 ts_tablet_manager.cc:1428] T 1f1f3a431e644d269c05d2513f9d99ab P a9814a8e125c4eeb8a479d67ef8b2045: Time spent starting tablet: real 0.012s user 0.010s sys 0.000s
I20260811 18:44:01.663254 26152 raft_consensus.cc:348] T 1f1f3a431e644d269c05d2513f9d99ab P 21050d666d634b858289431412d37063 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37699 } } peers { permanent_uuid: "21050d666d634b858289431412d37063" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34335 } } peers { permanent_uuid: "97fc6445d9244db78dd6284e19b22409" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33119 } }
I20260811 18:44:01.663710 26152 raft_consensus.cc:374] T 1f1f3a431e644d269c05d2513f9d99ab P 21050d666d634b858289431412d37063 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:44:01.663831 26152 raft_consensus.cc:729] T 1f1f3a431e644d269c05d2513f9d99ab P 21050d666d634b858289431412d37063 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21050d666d634b858289431412d37063, State: Initialized, Role: FOLLOWER
I20260811 18:44:01.664456 26152 consensus_queue.cc:260] T 1f1f3a431e644d269c05d2513f9d99ab P 21050d666d634b858289431412d37063 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37699 } } peers { permanent_uuid: "21050d666d634b858289431412d37063" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34335 } } peers { permanent_uuid: "97fc6445d9244db78dd6284e19b22409" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33119 } }
I20260811 18:44:01.665575 26110 heartbeater.cc:503] Master 127.30.185.190:46035 was elected leader, sending a full tablet report...
I20260811 18:44:01.665875 26155 raft_consensus.cc:348] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37699 } } peers { permanent_uuid: "21050d666d634b858289431412d37063" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34335 } } peers { permanent_uuid: "97fc6445d9244db78dd6284e19b22409" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33119 } }
I20260811 18:44:01.666320 26152 ts_tablet_manager.cc:1428] T 1f1f3a431e644d269c05d2513f9d99ab P 21050d666d634b858289431412d37063: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260811 18:44:01.666373 26155 raft_consensus.cc:374] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:44:01.666476 26155 raft_consensus.cc:729] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97fc6445d9244db78dd6284e19b22409, State: Initialized, Role: FOLLOWER
I20260811 18:44:01.667524 26155 consensus_queue.cc:260] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37699 } } peers { permanent_uuid: "21050d666d634b858289431412d37063" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34335 } } peers { permanent_uuid: "97fc6445d9244db78dd6284e19b22409" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33119 } }
I20260811 18:44:01.668762 25873 heartbeater.cc:503] Master 127.30.185.190:46035 was elected leader, sending a full tablet report...
I20260811 18:44:01.669517 26155 ts_tablet_manager.cc:1428] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260811 18:44:01.690476 26167 raft_consensus.cc:482] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:44:01.690769 26167 raft_consensus.cc:504] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37699 } } peers { permanent_uuid: "21050d666d634b858289431412d37063" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34335 } } peers { permanent_uuid: "97fc6445d9244db78dd6284e19b22409" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33119 } }
I20260811 18:44:01.692444 26167 leader_election.cc:288] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9814a8e125c4eeb8a479d67ef8b2045 (127.30.185.129:37699), 21050d666d634b858289431412d37063 (127.30.185.131:34335)
I20260811 18:44:01.701297 26148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:01.693088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:43645 (local address 127.30.185.131:34335)
0811 18:44:01.693567 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:01.693574 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:01.694169 (+ 595us) server_negotiation.cc:407] Connection header received
0811 18:44:01.694461 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:01.694469 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:01.694557 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:01.694657 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:44:01.695298 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.696356 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.698826 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.700070 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.700222 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:01.700299 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:01.700849 (+ 550us) server_negotiation.cc:299] Negotiation successful
0811 18:44:01.700980 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340}
I20260811 18:44:01.701282 26144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:01.693002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:46311 (local address 127.30.185.129:37699)
0811 18:44:01.693391 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:01.693401 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:01.693829 (+ 428us) server_negotiation.cc:407] Connection header received
0811 18:44:01.694078 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:01.694082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:01.694151 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:01.694253 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:44:01.694769 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.695932 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.698652 (+ 2720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.699968 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.700163 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:01.700240 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:01.700849 (+ 609us) server_negotiation.cc:299] Negotiation successful
0811 18:44:01.700980 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224}
I20260811 18:44:01.701861 26170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:01.692698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:37699 (local address 127.30.185.130:46311)
0811 18:44:01.693693 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:01.693709 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:01.693906 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:01.694278 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:01.694283 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:01.694309 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:44:01.694619 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:01.694627 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.696072 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.696076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:01.698460 (+ 2384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:01.698477 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.700251 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.700262 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:01.700582 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:01.700704 (+ 122us) client_negotiation.cc:769] Sending connection context
0811 18:44:01.700802 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 18:44:01.700971 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"mutex_wait_us":101,"thread_start_us":1069,"threads_started":1}
I20260811 18:44:01.701882 26168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:01.692735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:34335 (local address 127.30.185.130:43645)
0811 18:44:01.694055 (+ 1320us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:01.694069 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:01.694264 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:01.694673 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:01.694677 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:01.694702 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:44:01.695077 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:01.695093 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.696499 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.696504 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:01.698663 (+ 2159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:01.698677 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.700269 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.700274 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:01.700586 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:01.700702 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 18:44:01.700802 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 18:44:01.700958 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1224,"mutex_wait_us":126,"thread_start_us":541,"threads_started":1}
I20260811 18:44:01.703011 26032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f1f3a431e644d269c05d2513f9d99ab" candidate_uuid: "97fc6445d9244db78dd6284e19b22409" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21050d666d634b858289431412d37063" is_pre_election: true
I20260811 18:44:01.703015 25578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f1f3a431e644d269c05d2513f9d99ab" candidate_uuid: "97fc6445d9244db78dd6284e19b22409" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" is_pre_election: true
I20260811 18:44:01.703558 25578 raft_consensus.cc:2454] T 1f1f3a431e644d269c05d2513f9d99ab P a9814a8e125c4eeb8a479d67ef8b2045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97fc6445d9244db78dd6284e19b22409 in term 0.
I20260811 18:44:01.703553 26032 raft_consensus.cc:2454] T 1f1f3a431e644d269c05d2513f9d99ab P 21050d666d634b858289431412d37063 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97fc6445d9244db78dd6284e19b22409 in term 0.
I20260811 18:44:01.704267 25699 leader_election.cc:302] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 21050d666d634b858289431412d37063, 97fc6445d9244db78dd6284e19b22409, a9814a8e125c4eeb8a479d67ef8b2045; no voters:
I20260811 18:44:01.704696 26167 raft_consensus.cc:2781] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:44:01.704869 26167 raft_consensus.cc:482] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:44:01.704979 26167 raft_consensus.cc:3037] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:44:01.707204 26167 raft_consensus.cc:504] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37699 } } peers { permanent_uuid: "21050d666d634b858289431412d37063" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34335 } } peers { permanent_uuid: "97fc6445d9244db78dd6284e19b22409" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33119 } }
I20260811 18:44:01.707777 26167 leader_election.cc:288] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409 [CANDIDATE]: Term 1 election: Requested vote from peers a9814a8e125c4eeb8a479d67ef8b2045 (127.30.185.129:37699), 21050d666d634b858289431412d37063 (127.30.185.131:34335)
I20260811 18:44:01.708231 25578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f1f3a431e644d269c05d2513f9d99ab" candidate_uuid: "97fc6445d9244db78dd6284e19b22409" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9814a8e125c4eeb8a479d67ef8b2045"
I20260811 18:44:01.708258 26032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f1f3a431e644d269c05d2513f9d99ab" candidate_uuid: "97fc6445d9244db78dd6284e19b22409" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21050d666d634b858289431412d37063"
I20260811 18:44:01.708446 25578 raft_consensus.cc:3037] T 1f1f3a431e644d269c05d2513f9d99ab P a9814a8e125c4eeb8a479d67ef8b2045 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:44:01.708453 26032 raft_consensus.cc:3037] T 1f1f3a431e644d269c05d2513f9d99ab P 21050d666d634b858289431412d37063 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:44:01.710807 26032 raft_consensus.cc:2454] T 1f1f3a431e644d269c05d2513f9d99ab P 21050d666d634b858289431412d37063 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97fc6445d9244db78dd6284e19b22409 in term 1.
I20260811 18:44:01.711102 25578 raft_consensus.cc:2454] T 1f1f3a431e644d269c05d2513f9d99ab P a9814a8e125c4eeb8a479d67ef8b2045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97fc6445d9244db78dd6284e19b22409 in term 1.
I20260811 18:44:01.711225 25699 leader_election.cc:302] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21050d666d634b858289431412d37063, 97fc6445d9244db78dd6284e19b22409; no voters:
I20260811 18:44:01.711597 26167 raft_consensus.cc:2781] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:44:01.712052 26167 raft_consensus.cc:686] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409 [term 1 LEADER]: Becoming Leader. State: Replica: 97fc6445d9244db78dd6284e19b22409, State: Running, Role: LEADER
I20260811 18:44:01.712534 26167 consensus_queue.cc:237] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37699 } } peers { permanent_uuid: "21050d666d634b858289431412d37063" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34335 } } peers { permanent_uuid: "97fc6445d9244db78dd6284e19b22409" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33119 } }
I20260811 18:44:01.717247 25304 catalog_manager.cc:5495] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97fc6445d9244db78dd6284e19b22409 (127.30.185.130). New cstate: current_term: 1 leader_uuid: "97fc6445d9244db78dd6284e19b22409" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21050d666d634b858289431412d37063" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97fc6445d9244db78dd6284e19b22409" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33119 } health_report { overall_health: HEALTHY } } }
W20260811 18:44:01.727039 25875 tablet.cc:2369] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:44:01.754329 26150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:01.748097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49474 (local address 127.30.185.130:33119)
0811 18:44:01.748338 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:01.748346 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:01.748369 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:44:01.748474 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:01.748480 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:01.748577 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:01.748728 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:44:01.749089 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.750248 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.751690 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.752379 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.752502 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:01.752984 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:44:01.753690 (+ 706us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:44:01.753837 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:01.753931 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 18:44:01.754013 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 18:44:01.763041 26032 raft_consensus.cc:1261] T 1f1f3a431e644d269c05d2513f9d99ab P 21050d666d634b858289431412d37063 [term 1 FOLLOWER]: Refusing update from remote peer 97fc6445d9244db78dd6284e19b22409: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:44:01.763137 25578 raft_consensus.cc:1261] T 1f1f3a431e644d269c05d2513f9d99ab P a9814a8e125c4eeb8a479d67ef8b2045 [term 1 FOLLOWER]: Refusing update from remote peer 97fc6445d9244db78dd6284e19b22409: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:44:01.764148 26173 consensus_queue.cc:1035] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21050d666d634b858289431412d37063" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:44:01.764541 26167 consensus_queue.cc:1035] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:44:01.767535 26112 tablet.cc:2369] T 1f1f3a431e644d269c05d2513f9d99ab P 21050d666d634b858289431412d37063: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:44:01.776185 26176 mvcc.cc:205] Tried to move back new op lower bound from 7317396855851417600 to 7317396855658827776. Current Snapshot: MvccSnapshot[applied={T|T < 7317396855851417600}]
I20260811 18:44:01.776381 26178 mvcc.cc:205] Tried to move back new op lower bound from 7317396855851417600 to 7317396855658827776. Current Snapshot: MvccSnapshot[applied={T|T < 7317396855851417600}]
I20260811 18:44:01.792430 25304 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:44:01.792744 25304 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:44:01.796743 25304 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b: Sending DeleteTablet for 3 replicas of tablet 1f1f3a431e644d269c05d2513f9d99ab
I20260811 18:44:01.798034 25774 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f1f3a431e644d269c05d2513f9d99ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:44:01 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:49462
I20260811 18:44:01.798035 25996 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f1f3a431e644d269c05d2513f9d99ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:44:01 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:45278
I20260811 18:44:01.798117 25545 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f1f3a431e644d269c05d2513f9d99ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:44:01 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:56452
I20260811 18:44:01.799044 26204 tablet_replica.cc:330] T 1f1f3a431e644d269c05d2513f9d99ab P 21050d666d634b858289431412d37063: stopping tablet replica
I20260811 18:44:01.799070 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 25423
I20260811 18:44:01.799602 26204 raft_consensus.cc:2229] T 1f1f3a431e644d269c05d2513f9d99ab P 21050d666d634b858289431412d37063 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:44:01.800205 26204 raft_consensus.cc:2258] T 1f1f3a431e644d269c05d2513f9d99ab P 21050d666d634b858289431412d37063 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:44:01.801118 26202 tablet_replica.cc:330] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409: stopping tablet replica
I20260811 18:44:01.801378 26205 tablet_replica.cc:330] T 1f1f3a431e644d269c05d2513f9d99ab P a9814a8e125c4eeb8a479d67ef8b2045: stopping tablet replica
I20260811 18:44:01.801561 26202 raft_consensus.cc:2229] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:44:01.801868 26205 raft_consensus.cc:2229] T 1f1f3a431e644d269c05d2513f9d99ab P a9814a8e125c4eeb8a479d67ef8b2045 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:44:01.802305 26202 raft_consensus.cc:2258] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:44:01.802428 26205 raft_consensus.cc:2258] T 1f1f3a431e644d269c05d2513f9d99ab P a9814a8e125c4eeb8a479d67ef8b2045 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:44:01.804071 26202 ts_tablet_manager.cc:1905] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:44:01.804790 26205 ts_tablet_manager.cc:1905] T 1f1f3a431e644d269c05d2513f9d99ab P a9814a8e125c4eeb8a479d67ef8b2045: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:44:01.810325 26204 ts_tablet_manager.cc:1905] T 1f1f3a431e644d269c05d2513f9d99ab P 21050d666d634b858289431412d37063: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:44:01.811661 26202 ts_tablet_manager.cc:1918] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:44:01.811848 26202 log.cc:1196] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/1f1f3a431e644d269c05d2513f9d99ab
I20260811 18:44:01.812384 26202 ts_tablet_manager.cc:1939] T 1f1f3a431e644d269c05d2513f9d99ab P 97fc6445d9244db78dd6284e19b22409: Deleting consensus metadata
I20260811 18:44:01.813400 26205 ts_tablet_manager.cc:1918] T 1f1f3a431e644d269c05d2513f9d99ab P a9814a8e125c4eeb8a479d67ef8b2045: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:44:01.813572 26205 log.cc:1196] T 1f1f3a431e644d269c05d2513f9d99ab P a9814a8e125c4eeb8a479d67ef8b2045: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/wal/wals/1f1f3a431e644d269c05d2513f9d99ab
I20260811 18:44:01.814179 26205 ts_tablet_manager.cc:1939] T 1f1f3a431e644d269c05d2513f9d99ab P a9814a8e125c4eeb8a479d67ef8b2045: Deleting consensus metadata
I20260811 18:44:01.814196 25283 catalog_manager.cc:4841] TS 97fc6445d9244db78dd6284e19b22409 (127.30.185.130:33119): tablet 1f1f3a431e644d269c05d2513f9d99ab (table test-table [id=9e7c474f3123454282844fe39c6d31cf]) successfully deleted
I20260811 18:44:01.815960 25283 catalog_manager.cc:4841] TS a9814a8e125c4eeb8a479d67ef8b2045 (127.30.185.129:37699): tablet 1f1f3a431e644d269c05d2513f9d99ab (table test-table [id=9e7c474f3123454282844fe39c6d31cf]) successfully deleted
I20260811 18:44:01.817468 26204 ts_tablet_manager.cc:1918] T 1f1f3a431e644d269c05d2513f9d99ab P 21050d666d634b858289431412d37063: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:44:01.817654 26204 log.cc:1196] T 1f1f3a431e644d269c05d2513f9d99ab P 21050d666d634b858289431412d37063: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/1f1f3a431e644d269c05d2513f9d99ab
I20260811 18:44:01.818102 26204 ts_tablet_manager.cc:1939] T 1f1f3a431e644d269c05d2513f9d99ab P 21050d666d634b858289431412d37063: Deleting consensus metadata
I20260811 18:44:01.819861 25285 catalog_manager.cc:4841] TS 21050d666d634b858289431412d37063 (127.30.185.131:34335): tablet 1f1f3a431e644d269c05d2513f9d99ab (table test-table [id=9e7c474f3123454282844fe39c6d31cf]) successfully deleted
Aug 11 18:44:01 dist-test-slave-z8x9 krb5kdc[25226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.129@KRBTEST.COM
I20260811 18:44:01.830286 26144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:01.800423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56462 (local address 127.30.185.129:37699)
0811 18:44:01.800651 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:01.800661 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:01.812518 (+ 11857us) server_negotiation.cc:407] Connection header received
0811 18:44:01.812856 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:01.812862 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:01.812941 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:01.813026 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:01.815409 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.816571 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.818460 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.819100 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.819235 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:01.823673 (+ 4438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:01.823686 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:01.823701 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:01.823741 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:01.826953 (+ 3212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:01.827807 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:01.827814 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:01.827817 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:01.827960 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:01.828443 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:01.828454 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:01.828456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:01.828875 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:01.829321 (+ 446us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:01.829663 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 18:44:01.829893 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 18:44:01.831019 25642 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:44:01.922183 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 25423
I20260811 18:44:01.959937 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 25672
Aug 11 18:44:01 dist-test-slave-z8x9 krb5kdc[25226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.130@KRBTEST.COM
I20260811 18:44:01.984167 26150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:01.960795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49476 (local address 127.30.185.130:33119)
0811 18:44:01.960973 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:01.960977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:01.961087 (+ 110us) server_negotiation.cc:407] Connection header received
0811 18:44:01.961324 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:01.961328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:01.961414 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:01.961499 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:01.963337 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.964147 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.966243 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:01.966728 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:01.966838 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:01.977281 (+ 10443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:01.977295 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:01.977308 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:01.977351 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:01.980919 (+ 3568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:01.981778 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:01.981785 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:01.981794 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:01.981998 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:01.982425 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:01.982431 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:01.982441 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:01.982847 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:01.983173 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:01.983529 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 18:44:01.983746 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 18:44:01.984915 25863 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:44:02.100271 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 25672
I20260811 18:44:02.128824 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 25885
Aug 11 18:44:02 dist-test-slave-z8x9 krb5kdc[25226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.131@KRBTEST.COM
I20260811 18:44:02.144228 26148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:02.129672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45284 (local address 127.30.185.131:34335)
0811 18:44:02.129847 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:02.129862 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:02.129956 (+ 94us) server_negotiation.cc:407] Connection header received
0811 18:44:02.130099 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:02.130103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:02.130167 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:02.130251 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:02.131453 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:02.132164 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:02.133521 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:02.133918 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:02.134018 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:02.138945 (+ 4927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:02.138960 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:02.138977 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:02.139015 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:02.141830 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:02.142503 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:02.142510 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:02.142513 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:02.142650 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:02.142999 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:02.143003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:02.143004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:02.143350 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:02.143559 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:02.143828 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 18:44:02.143992 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 18:44:02.144949 26104 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:44:02.235517 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 25885
I20260811 18:44:02.265802 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 25254
I20260811 18:44:02.267046 25362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:44:02.382144 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 25254
I20260811 18:44:02.416050 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:46035
--webserver_interface=127.30.185.190
--webserver_port=43895
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:39755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:46035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:44:02.774186 26213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:44:02.774509 26213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:44:02.774603 26213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:44:02.781983 26213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:44:02.782111 26213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:44:02.782167 26213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:44:02.782231 26213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:44:02.782277 26213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:44:02.792521 26213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:39755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:46035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:46035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=43895
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:44:02.793877 26213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:44:02.795250 26213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:44:02.824762 26221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:02.824878 26227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:02.824793 26222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:44:02.825773 26213 server_base.cc:1034] running on GCE node
I20260811 18:44:02.826799 26213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:44:02.828655 26213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:44:02.829939 26213 hybrid_clock.cc:648] HybridClock initialized: now 1786473842829870 us; error 66 us; skew 500 ppm
I20260811 18:44:02.830443 26213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:44:02 dist-test-slave-z8x9 krb5kdc[25226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473842, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:44:02.840258 26213 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:44:02.842372 26213 webserver.cc:460] Webserver started at http://127.30.185.190:43895/ using document root <none> and password file <none>
I20260811 18:44:02.843240 26213 fs_manager.cc:359] Metadata directory not provided
I20260811 18:44:02.843429 26213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:44:02.850034 26213 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 18:44:02.853699 26238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:02.855118 26213 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:44:02.855329 26213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "0d1f7192ff6e4019b62175cfae0a551b"
format_stamp: "Formatted at 2026-08-11 18:43:59 on dist-test-slave-z8x9"
server_key: "bb7590aea56c12beca5a0b2dfaf0c441"
server_key_iv: "317c63af76dcc94e7eca65740af8bb42"
server_key_version: "encryptionkey@0"
I20260811 18:44:02.856516 26213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:44:02.876932 26213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:44:02.882119 26213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:44:02.882448 26213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:44:02.904464 26213 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:46035
I20260811 18:44:02.904487 26345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:46035 every 8 connection(s)
I20260811 18:44:02.906247 26213 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:44:02.908460 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 26213
I20260811 18:44:02.909577 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:37699
--local_ip_for_outbound_sockets=127.30.185.129
--tserver_master_addrs=127.30.185.190:46035
--webserver_port=44951
--webserver_interface=127.30.185.129
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:39755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:44:02.916841 26347 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:44:02.922901 26347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b: Bootstrap starting.
I20260811 18:44:02.943807 26347 log.cc:824] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:44:02.958212 26347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:44:02.958851 26347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b: Bootstrap complete.
I20260811 18:44:02.969290 26347 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1f7192ff6e4019b62175cfae0a551b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46035 } }
I20260811 18:44:02.970266 26347 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d1f7192ff6e4019b62175cfae0a551b, State: Initialized, Role: FOLLOWER
I20260811 18:44:02.971099 26347 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1f7192ff6e4019b62175cfae0a551b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46035 } }
I20260811 18:44:02.971388 26347 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:44:02.971527 26347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:44:02.971732 26347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:44:02.976116 26347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1f7192ff6e4019b62175cfae0a551b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46035 } }
I20260811 18:44:02.976801 26347 leader_election.cc:302] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d1f7192ff6e4019b62175cfae0a551b; no voters:
I20260811 18:44:02.977615 26347 leader_election.cc:288] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 18:44:02.977784 26354 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:44:02.979126 26354 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 2 LEADER]: Becoming Leader. State: Replica: 0d1f7192ff6e4019b62175cfae0a551b, State: Running, Role: LEADER
I20260811 18:44:02.979763 26354 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1f7192ff6e4019b62175cfae0a551b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46035 } }
I20260811 18:44:02.980201 26347 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:44:02.982574 26356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d1f7192ff6e4019b62175cfae0a551b. Latest consensus state: current_term: 2 leader_uuid: "0d1f7192ff6e4019b62175cfae0a551b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1f7192ff6e4019b62175cfae0a551b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46035 } } }
I20260811 18:44:02.983114 26356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [sys.catalog]: This master's current role is: LEADER
I20260811 18:44:02.983778 26355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0d1f7192ff6e4019b62175cfae0a551b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1f7192ff6e4019b62175cfae0a551b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46035 } } }
I20260811 18:44:02.984122 26355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [sys.catalog]: This master's current role is: LEADER
I20260811 18:44:02.988010 26362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:44:02.999293 26362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:44:03.001473 26362 catalog_manager.cc:1259] Loaded cluster ID: bb74d411909049e09385a3e96c736e18
I20260811 18:44:03.001628 26362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:44:03.006209 26362 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 18:44:03.037160 26362 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 18:44:03.062945 26362 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:44:03.065440 26362 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b: Loaded TSK: 0
I20260811 18:44:03.067776 26362 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 18:44:03.314532 26349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:44:03.314908 26349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:44:03.315083 26349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:44:03.323017 26349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:44:03.323185 26349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:44:03.323366 26349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:44:03.333896 26349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:39755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:37699
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=44951
--tserver_master_addrs=127.30.185.190:46035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:44:03.335285 26349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:44:03.336659 26349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:44:03.352942 26400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:03.352932 26402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:03.357532 26398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:44:03.359026 26349 server_base.cc:1034] running on GCE node
I20260811 18:44:03.359766 26349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:44:03.360909 26349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:44:03.362156 26349 hybrid_clock.cc:648] HybridClock initialized: now 1786473843362100 us; error 47 us; skew 500 ppm
I20260811 18:44:03.362548 26349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:44:03 dist-test-slave-z8x9 krb5kdc[25226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473843, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:44:03.366207 26349 init.cc:377] Logged in from keytab as kudu/127.30.185.129@KRBTEST.COM (short username kudu)
I20260811 18:44:03.368042 26349 webserver.cc:460] Webserver started at http://127.30.185.129:44951/ using document root <none> and password file <none>
I20260811 18:44:03.368798 26349 fs_manager.cc:359] Metadata directory not provided
I20260811 18:44:03.368973 26349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:44:03.374884 26349 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:44:03.378136 26414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:03.379675 26349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:44:03.379858 26349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "a9814a8e125c4eeb8a479d67ef8b2045"
format_stamp: "Formatted at 2026-08-11 18:44:00 on dist-test-slave-z8x9"
server_key: "bcc3bb38a8b883c752bcced5e84fdab6"
server_key_iv: "692178b074c55d6786d25c63b6a9f336"
server_key_version: "encryptionkey@0"
I20260811 18:44:03.380960 26349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:44:03.409067 26349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:44:03.413568 26349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:44:03.413909 26349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:44:03.414914 26349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:44:03.416287 26349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:44:03.416388 26349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:03.416471 26349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:44:03.416527 26349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:03.451982 26349 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:37699
I20260811 18:44:03.452090 26606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:37699 every 8 connection(s)
I20260811 18:44:03.453843 26349 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:44:03.455259 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 26349
I20260811 18:44:03.455955 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:33119
--local_ip_for_outbound_sockets=127.30.185.130
--tserver_master_addrs=127.30.185.190:46035
--webserver_port=43679
--webserver_interface=127.30.185.130
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:39755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:44:03 dist-test-slave-z8x9 krb5kdc[25226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473843, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:44:03.480690 26611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:03.457227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:37039 (local address 127.30.185.190:46035)
0811 18:44:03.457981 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:03.458004 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:03.458826 (+ 822us) server_negotiation.cc:407] Connection header received
0811 18:44:03.461271 (+ 2445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:03.461294 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:03.461787 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:03.462204 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:03.463647 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:03.465740 (+ 2093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:03.467333 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:03.468324 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:03.468544 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:03.472532 (+ 3988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:03.472541 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:03.472557 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:03.472597 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:03.475713 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:03.476587 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:03.476596 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:03.476602 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:03.476777 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:03.477300 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:03.477306 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:03.477311 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:03.477835 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:03.478055 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:03.478871 (+ 816us) server_negotiation.cc:299] Negotiation successful
0811 18:44:03.479279 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":279,"threads_started":1}
I20260811 18:44:03.480705 26615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:03.457413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.129:37039)
0811 18:44:03.458604 (+ 1191us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:03.458662 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:03.460549 (+ 1887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:03.462324 (+ 1775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:03.462332 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:03.462867 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:44:03.463396 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:03.463413 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:03.465912 (+ 2499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:03.465916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:03.467169 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:03.467181 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:03.468587 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:03.468595 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:03.468891 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:03.469667 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:44:03.469691 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:44:03.472208 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:44:03.475910 (+ 3702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:03.475919 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:03.475936 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:03.476424 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:03.476917 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:03.476922 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:03.476926 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:03.477147 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:03.478005 (+ 858us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:44:03.478014 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:44:03.478373 (+ 359us) client_negotiation.cc:769] Sending connection context
0811 18:44:03.478647 (+ 274us) client_negotiation.cc:240] Negotiation successful
0811 18:44:03.479260 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":904,"thread_start_us":664,"threads_started":1}
I20260811 18:44:03.483743 26607 heartbeater.cc:347] Connected to a master server at 127.30.185.190:46035
I20260811 18:44:03.484143 26607 heartbeater.cc:464] Registering TS with master...
I20260811 18:44:03.485271 26607 heartbeater.cc:511] Master 127.30.185.190:46035 requested a full tablet report, sending...
I20260811 18:44:03.487957 26266 ts_manager.cc:194] Registered new tserver with Master: a9814a8e125c4eeb8a479d67ef8b2045 (127.30.185.129:37699)
I20260811 18:44:03.489565 26266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:37039
W20260811 18:44:03.811858 26616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:44:03.812297 26616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:44:03.812494 26616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:44:03.820372 26616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:44:03.820552 26616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:44:03.820752 26616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:44:03.831656 26616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:39755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:33119
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=43679
--tserver_master_addrs=127.30.185.190:46035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:44:03.833142 26616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:44:03.834734 26616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:44:03.849401 26625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:03.849498 26635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:03.849428 26627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:44:03.850402 26616 server_base.cc:1034] running on GCE node
I20260811 18:44:03.851181 26616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:44:03.852373 26616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:44:03.853662 26616 hybrid_clock.cc:648] HybridClock initialized: now 1786473843853606 us; error 83 us; skew 500 ppm
I20260811 18:44:03.854068 26616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:44:03 dist-test-slave-z8x9 krb5kdc[25226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473843, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:44:03.857869 26616 init.cc:377] Logged in from keytab as kudu/127.30.185.130@KRBTEST.COM (short username kudu)
I20260811 18:44:03.859575 26616 webserver.cc:460] Webserver started at http://127.30.185.130:43679/ using document root <none> and password file <none>
I20260811 18:44:03.860374 26616 fs_manager.cc:359] Metadata directory not provided
I20260811 18:44:03.860560 26616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:44:03.866686 26616 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:44:03.869902 26643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:03.871327 26616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:44:03.871546 26616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "97fc6445d9244db78dd6284e19b22409"
format_stamp: "Formatted at 2026-08-11 18:44:00 on dist-test-slave-z8x9"
server_key: "b74b4025ff34bc4a1f221c5b346f0227"
server_key_iv: "48f52eccb0e58911547dcfa730d986f3"
server_key_version: "encryptionkey@0"
I20260811 18:44:03.872637 26616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:44:03.905623 26616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:44:03.910344 26616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:44:03.910640 26616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:44:03.911700 26616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:44:03.913120 26616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:44:03.913282 26616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 18:44:03.913421 26616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:44:03.913519 26616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:03.957788 26616 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:33119
I20260811 18:44:03.957897 26855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:33119 every 8 connection(s)
I20260811 18:44:03.959941 26616 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:44:03.969825 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 26616
I20260811 18:44:03.970603 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:34335
--local_ip_for_outbound_sockets=127.30.185.131
--tserver_master_addrs=127.30.185.190:46035
--webserver_port=40431
--webserver_interface=127.30.185.131
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:39755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:44:03 dist-test-slave-z8x9 krb5kdc[25226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473843, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:44:03.983299 26611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:03.962905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:59083 (local address 127.30.185.190:46035)
0811 18:44:03.963126 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:03.963132 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:03.964390 (+ 1258us) server_negotiation.cc:407] Connection header received
0811 18:44:03.966334 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:03.966339 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:03.966434 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:03.966544 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:03.968147 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:03.968966 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:03.971486 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:03.971914 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:03.972012 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:03.976515 (+ 4503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:03.976521 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:03.976524 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:03.976560 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:03.979383 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:03.980187 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:03.980191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:03.980194 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:03.980340 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:03.980905 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:03.980912 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:03.980918 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:03.981401 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:03.981563 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:03.982535 (+ 972us) server_negotiation.cc:299] Negotiation successful
0811 18:44:03.982821 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 18:44:03.984303 26858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:03.963410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.130:59083)
0811 18:44:03.964183 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:03.964255 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:03.966041 (+ 1786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:03.966780 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:03.966788 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:03.967382 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:44:03.967954 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:03.967970 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:03.969125 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:03.969129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:03.971326 (+ 2197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:03.971345 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:03.972184 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:03.972196 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:03.972557 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:03.973418 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:44:03.973452 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:44:03.976230 (+ 2778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:44:03.979557 (+ 3327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:03.979566 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:03.979584 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:03.980038 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:03.980462 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:03.980467 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:03.980470 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:03.980737 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:03.981551 (+ 814us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:44:03.981558 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:44:03.982095 (+ 537us) client_negotiation.cc:769] Sending connection context
0811 18:44:03.982389 (+ 294us) client_negotiation.cc:240] Negotiation successful
0811 18:44:03.982973 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":314,"threads_started":1}
I20260811 18:44:03.986071 26856 heartbeater.cc:347] Connected to a master server at 127.30.185.190:46035
I20260811 18:44:03.986459 26856 heartbeater.cc:464] Registering TS with master...
I20260811 18:44:03.987216 26856 heartbeater.cc:511] Master 127.30.185.190:46035 requested a full tablet report, sending...
I20260811 18:44:03.988751 26266 ts_manager.cc:194] Registered new tserver with Master: 97fc6445d9244db78dd6284e19b22409 (127.30.185.130:33119)
I20260811 18:44:03.990075 26266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:59083
W20260811 18:44:04.322310 26863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:44:04.322628 26863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:44:04.322727 26863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:44:04.330547 26863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:44:04.330681 26863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:44:04.330797 26863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:44:04.340735 26863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:39755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:34335
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=40431
--tserver_master_addrs=127.30.185.190:46035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:44:04.342162 26863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:44:04.343600 26863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:44:04.357476 26872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:04.357508 26870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:04.357801 26877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:44:04.359016 26863 server_base.cc:1034] running on GCE node
I20260811 18:44:04.359684 26863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:44:04.360813 26863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:44:04.362028 26863 hybrid_clock.cc:648] HybridClock initialized: now 1786473844361964 us; error 64 us; skew 500 ppm
I20260811 18:44:04.362377 26863 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:44:04 dist-test-slave-z8x9 krb5kdc[25226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473844, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:44:04.365753 26863 init.cc:377] Logged in from keytab as kudu/127.30.185.131@KRBTEST.COM (short username kudu)
I20260811 18:44:04.367414 26863 webserver.cc:460] Webserver started at http://127.30.185.131:40431/ using document root <none> and password file <none>
I20260811 18:44:04.368176 26863 fs_manager.cc:359] Metadata directory not provided
I20260811 18:44:04.368355 26863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:44:04.374152 26863 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:44:04.377089 26895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:04.378582 26863 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:44:04.378782 26863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "21050d666d634b858289431412d37063"
format_stamp: "Formatted at 2026-08-11 18:44:01 on dist-test-slave-z8x9"
server_key: "99c07aea91736332afe04d955b569409"
server_key_iv: "05e37c42b36a428bd37d3dc2190a60d9"
server_key_version: "encryptionkey@0"
I20260811 18:44:04.379819 26863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:44:04.400876 26863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:44:04.405473 26863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:44:04.405777 26863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:44:04.406834 26863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:44:04.408299 26863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:44:04.408428 26863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:04.408576 26863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:44:04.408666 26863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:04.443897 26863 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:34335
I20260811 18:44:04.443945 27088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:34335 every 8 connection(s)
I20260811 18:44:04.445550 26863 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:44:04.455938 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 26863
Aug 11 18:44:04 dist-test-slave-z8x9 krb5kdc[25226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473844, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:44:04.467727 26611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:04.448329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:48753 (local address 127.30.185.190:46035)
0811 18:44:04.448521 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:04.448526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:04.449795 (+ 1269us) server_negotiation.cc:407] Connection header received
0811 18:44:04.451720 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:04.451728 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:04.451841 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:04.451959 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:04.453699 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.454432 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.456325 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.456744 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.456894 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:04.460742 (+ 3848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:04.460748 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:04.460753 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:04.460790 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:04.464074 (+ 3284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:04.465052 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:04.465056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:04.465059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:04.465232 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:04.465764 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:04.465767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:04.465769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:04.466031 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:04.466140 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:04.467048 (+ 908us) server_negotiation.cc:299] Negotiation successful
0811 18:44:04.467391 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 18:44:04.468791 27092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:04.448779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.131:48753)
0811 18:44:04.449608 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:04.449660 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:04.451399 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:04.452237 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:04.452245 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:04.452755 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:44:04.453507 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:04.453526 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.454600 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.454604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:04.456177 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:04.456194 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.457034 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.457044 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:04.457480 (+ 436us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:04.458262 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:44:04.458291 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:44:04.460517 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:44:04.464225 (+ 3708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:04.464235 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:04.464254 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:04.464916 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:04.465346 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:04.465352 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:04.465357 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:04.465631 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:04.466157 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:44:04.466166 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:44:04.466628 (+ 462us) client_negotiation.cc:769] Sending connection context
0811 18:44:04.466914 (+ 286us) client_negotiation.cc:240] Negotiation successful
0811 18:44:04.467314 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":386,"threads_started":1}
I20260811 18:44:04.470534 27090 heartbeater.cc:347] Connected to a master server at 127.30.185.190:46035
I20260811 18:44:04.470902 27090 heartbeater.cc:464] Registering TS with master...
I20260811 18:44:04.471951 27090 heartbeater.cc:511] Master 127.30.185.190:46035 requested a full tablet report, sending...
I20260811 18:44:04.473376 27094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:04.456957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47198 (local address 127.30.185.190:46035)
0811 18:44:04.457546 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:04.457553 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:04.457878 (+ 325us) server_negotiation.cc:407] Connection header received
0811 18:44:04.458084 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:04.458096 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:04.458154 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:04.458299 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:04.459847 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.460835 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.462558 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.463317 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.463455 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:04.466770 (+ 3315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:04.466775 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:04.466778 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:04.466805 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:04.470151 (+ 3346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:04.471021 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:04.471028 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:04.471033 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:04.471202 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:04.471611 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:04.471616 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:04.471619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:04.472006 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:04.472278 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:04.472661 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 18:44:04.472908 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":324,"threads_started":1}
I20260811 18:44:04.473793 26266 ts_manager.cc:194] Registered new tserver with Master: 21050d666d634b858289431412d37063 (127.30.185.131:34335)
I20260811 18:44:04.474939 26266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:48753
I20260811 18:44:04.476312 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:44:04.483460 27094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:04.477399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47214 (local address 127.30.185.190:46035)
0811 18:44:04.477637 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:04.477646 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:04.478061 (+ 415us) server_negotiation.cc:407] Connection header received
0811 18:44:04.478234 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:04.478237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:04.478359 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:04.478487 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:44:04.478872 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.479578 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.481117 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.481681 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.481782 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:04.482334 (+ 552us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:44:04.482839 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:44:04.483013 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:04.483085 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 18:44:04.483171 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 18:44:04.484579 26264 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:47214:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:44:04.492166 26264 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:44:04.493338 26607 heartbeater.cc:503] Master 127.30.185.190:46035 was elected leader, sending a full tablet report...
I20260811 18:44:04.515617 27108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:04.504953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:33119 (local address 127.0.0.1:58364)
0811 18:44:04.505861 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:04.505900 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:04.506096 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:04.507149 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:04.507155 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:04.507192 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:44:04.507664 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:04.507681 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.509673 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.509679 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:04.512186 (+ 2507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:04.512205 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.514462 (+ 2257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.514469 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:04.514674 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:04.514734 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 18:44:04.514889 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 18:44:04.515118 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"mutex_wait_us":1,"thread_start_us":421,"threads_started":1}
I20260811 18:44:04.516659 27109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:04.505151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58364 (local address 127.30.185.130:33119)
0811 18:44:04.505773 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:04.505783 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:04.506001 (+ 218us) server_negotiation.cc:407] Connection header received
0811 18:44:04.506357 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:04.506369 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:04.506657 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:04.506862 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:44:04.507884 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.509501 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.512381 (+ 2880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.513954 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.514116 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:04.514299 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:04.516101 (+ 1802us) server_negotiation.cc:299] Negotiation successful
0811 18:44:04.516295 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":287,"threads_started":1}
I20260811 18:44:04.516651 27111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:04.506247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:37699 (local address 127.0.0.1:47892)
0811 18:44:04.506822 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:04.506841 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:04.506963 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:04.508737 (+ 1774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:04.508741 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:04.508766 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:44:04.509110 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:04.509121 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.511146 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.511151 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:04.513281 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:04.513298 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.515504 (+ 2206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.515511 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:04.516014 (+ 503us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:04.516052 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 18:44:04.516140 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 18:44:04.516270 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":313,"threads_started":1}
I20260811 18:44:04.517733 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:04.506656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47892 (local address 127.30.185.129:37699)
0811 18:44:04.507937 (+ 1281us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:04.507948 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:04.507976 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:44:04.508077 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:04.508088 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:04.508308 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:04.508518 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:44:04.509342 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.510968 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.513487 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.514972 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.515160 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:04.515328 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:04.517220 (+ 1892us) server_negotiation.cc:299] Negotiation successful
0811 18:44:04.517403 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1112,"thread_start_us":942,"threads_started":1}
I20260811 18:44:04.521155 26729 tablet_service.cc:1467] Processing CreateTablet for tablet ed54559ac5d94f6d987221ab32083a04 (DEFAULT_TABLE table=test-table [id=7553ce26cbf24a678c25ce905265aad8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:44:04.521965 26495 tablet_service.cc:1467] Processing CreateTablet for tablet ed54559ac5d94f6d987221ab32083a04 (DEFAULT_TABLE table=test-table [id=7553ce26cbf24a678c25ce905265aad8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:44:04.522794 26729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed54559ac5d94f6d987221ab32083a04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:44:04.523403 26495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed54559ac5d94f6d987221ab32083a04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:44:04.526892 27114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:04.507086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:34335 (local address 127.0.0.1:48164)
0811 18:44:04.517796 (+ 10710us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:04.517817 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:04.517953 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:04.518970 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:04.518975 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:04.519005 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:44:04.519817 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:04.519830 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.521725 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.521733 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:04.523916 (+ 2183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:04.523934 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.525956 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.525967 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:04.526158 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:04.526198 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 18:44:04.526373 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 18:44:04.526521 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10581,"thread_start_us":10834,"threads_started":1}
I20260811 18:44:04.526921 27115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:04.507766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48164 (local address 127.30.185.131:34335)
0811 18:44:04.517845 (+ 10079us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:04.517856 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:04.517893 (+ 37us) server_negotiation.cc:407] Connection header received
0811 18:44:04.518180 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:04.518191 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:04.518732 (+ 541us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:04.519848 (+ 1116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:44:04.520123 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.521555 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.524150 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.525470 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.525687 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:04.525946 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:04.526422 (+ 476us) server_negotiation.cc:299] Negotiation successful
0811 18:44:04.526584 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9870,"thread_start_us":9293,"threads_started":1}
I20260811 18:44:04.530638 26969 tablet_service.cc:1467] Processing CreateTablet for tablet ed54559ac5d94f6d987221ab32083a04 (DEFAULT_TABLE table=test-table [id=7553ce26cbf24a678c25ce905265aad8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:44:04.531935 26969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed54559ac5d94f6d987221ab32083a04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:44:04.539312 27119 tablet_bootstrap.cc:492] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409: Bootstrap starting.
I20260811 18:44:04.540154 27116 tablet_bootstrap.cc:492] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045: Bootstrap starting.
I20260811 18:44:04.543308 27119 tablet_bootstrap.cc:654] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409: Neither blocks nor log segments found. Creating new log.
I20260811 18:44:04.543916 27116 tablet_bootstrap.cc:654] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045: Neither blocks nor log segments found. Creating new log.
I20260811 18:44:04.544461 27119 log.cc:824] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409: Log is configured to *not* fsync() on all Append() calls
I20260811 18:44:04.545251 27116 log.cc:824] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045: Log is configured to *not* fsync() on all Append() calls
I20260811 18:44:04.546223 27125 tablet_bootstrap.cc:492] T ed54559ac5d94f6d987221ab32083a04 P 21050d666d634b858289431412d37063: Bootstrap starting.
I20260811 18:44:04.551242 27125 tablet_bootstrap.cc:654] T ed54559ac5d94f6d987221ab32083a04 P 21050d666d634b858289431412d37063: Neither blocks nor log segments found. Creating new log.
I20260811 18:44:04.553359 27125 log.cc:824] T ed54559ac5d94f6d987221ab32083a04 P 21050d666d634b858289431412d37063: Log is configured to *not* fsync() on all Append() calls
I20260811 18:44:04.554397 27119 tablet_bootstrap.cc:492] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409: No bootstrap required, opened a new log
I20260811 18:44:04.554526 27116 tablet_bootstrap.cc:492] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045: No bootstrap required, opened a new log
I20260811 18:44:04.554716 27119 ts_tablet_manager.cc:1397] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.004s
I20260811 18:44:04.554862 27116 ts_tablet_manager.cc:1397] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.004s
I20260811 18:44:04.556552 27125 tablet_bootstrap.cc:492] T ed54559ac5d94f6d987221ab32083a04 P 21050d666d634b858289431412d37063: No bootstrap required, opened a new log
I20260811 18:44:04.556838 27125 ts_tablet_manager.cc:1397] T ed54559ac5d94f6d987221ab32083a04 P 21050d666d634b858289431412d37063: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 18:44:04.562721 27119 raft_consensus.cc:348] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97fc6445d9244db78dd6284e19b22409" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33119 } } peers { permanent_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37699 } } peers { permanent_uuid: "21050d666d634b858289431412d37063" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34335 } }
I20260811 18:44:04.563125 27119 raft_consensus.cc:374] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:44:04.563227 27119 raft_consensus.cc:729] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97fc6445d9244db78dd6284e19b22409, State: Initialized, Role: FOLLOWER
I20260811 18:44:04.563881 27119 consensus_queue.cc:260] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97fc6445d9244db78dd6284e19b22409" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33119 } } peers { permanent_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37699 } } peers { permanent_uuid: "21050d666d634b858289431412d37063" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34335 } }
I20260811 18:44:04.565071 26856 heartbeater.cc:503] Master 127.30.185.190:46035 was elected leader, sending a full tablet report...
I20260811 18:44:04.565650 27119 ts_tablet_manager.cc:1428] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260811 18:44:04.567145 27116 raft_consensus.cc:348] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97fc6445d9244db78dd6284e19b22409" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33119 } } peers { permanent_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37699 } } peers { permanent_uuid: "21050d666d634b858289431412d37063" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34335 } }
I20260811 18:44:04.567638 27116 raft_consensus.cc:374] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:44:04.567761 27116 raft_consensus.cc:729] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9814a8e125c4eeb8a479d67ef8b2045, State: Initialized, Role: FOLLOWER
I20260811 18:44:04.567960 27125 raft_consensus.cc:348] T ed54559ac5d94f6d987221ab32083a04 P 21050d666d634b858289431412d37063 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97fc6445d9244db78dd6284e19b22409" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33119 } } peers { permanent_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37699 } } peers { permanent_uuid: "21050d666d634b858289431412d37063" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34335 } }
I20260811 18:44:04.568408 27125 raft_consensus.cc:374] T ed54559ac5d94f6d987221ab32083a04 P 21050d666d634b858289431412d37063 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:44:04.568536 27125 raft_consensus.cc:729] T ed54559ac5d94f6d987221ab32083a04 P 21050d666d634b858289431412d37063 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21050d666d634b858289431412d37063, State: Initialized, Role: FOLLOWER
I20260811 18:44:04.568420 27116 consensus_queue.cc:260] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97fc6445d9244db78dd6284e19b22409" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33119 } } peers { permanent_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37699 } } peers { permanent_uuid: "21050d666d634b858289431412d37063" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34335 } }
I20260811 18:44:04.569211 27125 consensus_queue.cc:260] T ed54559ac5d94f6d987221ab32083a04 P 21050d666d634b858289431412d37063 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97fc6445d9244db78dd6284e19b22409" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33119 } } peers { permanent_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37699 } } peers { permanent_uuid: "21050d666d634b858289431412d37063" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34335 } }
I20260811 18:44:04.570345 27116 ts_tablet_manager.cc:1428] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045: Time spent starting tablet: real 0.015s user 0.008s sys 0.006s
I20260811 18:44:04.571183 27090 heartbeater.cc:503] Master 127.30.185.190:46035 was elected leader, sending a full tablet report...
I20260811 18:44:04.571806 27125 ts_tablet_manager.cc:1428] T ed54559ac5d94f6d987221ab32083a04 P 21050d666d634b858289431412d37063: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
W20260811 18:44:04.697362 27091 tablet.cc:2369] T ed54559ac5d94f6d987221ab32083a04 P 21050d666d634b858289431412d37063: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:44:04.704578 27129 raft_consensus.cc:482] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:44:04.704950 27129 raft_consensus.cc:504] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97fc6445d9244db78dd6284e19b22409" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33119 } } peers { permanent_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37699 } } peers { permanent_uuid: "21050d666d634b858289431412d37063" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34335 } }
W20260811 18:44:04.706453 26610 tablet.cc:2369] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:44:04.706924 27129 leader_election.cc:288] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9814a8e125c4eeb8a479d67ef8b2045 (127.30.185.129:37699), 21050d666d634b858289431412d37063 (127.30.185.131:34335)
W20260811 18:44:04.712284 26857 tablet.cc:2369] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:44:04.715366 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:04.707556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:44675 (local address 127.30.185.129:37699)
0811 18:44:04.707808 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:04.707817 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:04.708650 (+ 833us) server_negotiation.cc:407] Connection header received
0811 18:44:04.708885 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:04.708894 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:04.708989 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:04.709058 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:44:04.709777 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.710554 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.712732 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.714180 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.714379 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:04.714469 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:04.714926 (+ 457us) server_negotiation.cc:299] Negotiation successful
0811 18:44:04.715075 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 18:44:04.715380 27115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:04.707581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:43013 (local address 127.30.185.131:34335)
0811 18:44:04.707841 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:04.707850 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:04.708756 (+ 906us) server_negotiation.cc:407] Connection header received
0811 18:44:04.708996 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:04.709001 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:04.709105 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:04.709235 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:44:04.709923 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.710820 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.712893 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.714307 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.714465 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:04.714554 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:04.714925 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 18:44:04.715075 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 18:44:04.715621 27134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:04.707413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:34335 (local address 127.30.185.130:43013)
0811 18:44:04.708587 (+ 1174us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:04.708628 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:04.708839 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:04.709269 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:04.709273 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:04.709305 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:44:04.709755 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:04.709768 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.710964 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.710968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:04.712724 (+ 1756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:04.712740 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.714505 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.714510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:04.714703 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:04.714770 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 18:44:04.714874 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 18:44:04.715115 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1054,"mutex_wait_us":55,"thread_start_us":680,"threads_started":1}
I20260811 18:44:04.715680 27135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:04.707133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:37699 (local address 127.30.185.130:44675)
0811 18:44:04.708512 (+ 1379us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:04.708530 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:04.708715 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:04.709110 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:04.709115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:04.709144 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:44:04.709642 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:04.709653 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.710691 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.710697 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:04.712546 (+ 1849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:04.712565 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.714429 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.714438 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:04.714697 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:04.714773 (+ 76us) client_negotiation.cc:769] Sending connection context
0811 18:44:04.714877 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 18:44:04.715115 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1245,"mutex_wait_us":164,"thread_start_us":1031,"threads_started":1}
I20260811 18:44:04.716869 26525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed54559ac5d94f6d987221ab32083a04" candidate_uuid: "97fc6445d9244db78dd6284e19b22409" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" is_pre_election: true
I20260811 18:44:04.716867 27007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed54559ac5d94f6d987221ab32083a04" candidate_uuid: "97fc6445d9244db78dd6284e19b22409" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21050d666d634b858289431412d37063" is_pre_election: true
I20260811 18:44:04.717471 27007 raft_consensus.cc:2454] T ed54559ac5d94f6d987221ab32083a04 P 21050d666d634b858289431412d37063 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97fc6445d9244db78dd6284e19b22409 in term 0.
I20260811 18:44:04.717470 26525 raft_consensus.cc:2454] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97fc6445d9244db78dd6284e19b22409 in term 0.
I20260811 18:44:04.718174 26644 leader_election.cc:302] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97fc6445d9244db78dd6284e19b22409, a9814a8e125c4eeb8a479d67ef8b2045; no voters:
I20260811 18:44:04.718600 27129 raft_consensus.cc:2781] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:44:04.718767 27129 raft_consensus.cc:482] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:44:04.718919 27129 raft_consensus.cc:3037] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:44:04.721400 27129 raft_consensus.cc:504] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97fc6445d9244db78dd6284e19b22409" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33119 } } peers { permanent_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37699 } } peers { permanent_uuid: "21050d666d634b858289431412d37063" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34335 } }
I20260811 18:44:04.722093 27129 leader_election.cc:288] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409 [CANDIDATE]: Term 1 election: Requested vote from peers a9814a8e125c4eeb8a479d67ef8b2045 (127.30.185.129:37699), 21050d666d634b858289431412d37063 (127.30.185.131:34335)
I20260811 18:44:04.722564 26525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed54559ac5d94f6d987221ab32083a04" candidate_uuid: "97fc6445d9244db78dd6284e19b22409" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9814a8e125c4eeb8a479d67ef8b2045"
I20260811 18:44:04.722638 27007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed54559ac5d94f6d987221ab32083a04" candidate_uuid: "97fc6445d9244db78dd6284e19b22409" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21050d666d634b858289431412d37063"
I20260811 18:44:04.722802 26525 raft_consensus.cc:3037] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:44:04.722833 27007 raft_consensus.cc:3037] T ed54559ac5d94f6d987221ab32083a04 P 21050d666d634b858289431412d37063 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:44:04.726166 27007 raft_consensus.cc:2454] T ed54559ac5d94f6d987221ab32083a04 P 21050d666d634b858289431412d37063 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97fc6445d9244db78dd6284e19b22409 in term 1.
I20260811 18:44:04.726166 26525 raft_consensus.cc:2454] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97fc6445d9244db78dd6284e19b22409 in term 1.
I20260811 18:44:04.726649 26644 leader_election.cc:302] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97fc6445d9244db78dd6284e19b22409, a9814a8e125c4eeb8a479d67ef8b2045; no voters:
I20260811 18:44:04.726989 27129 raft_consensus.cc:2781] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:44:04.727499 27129 raft_consensus.cc:686] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409 [term 1 LEADER]: Becoming Leader. State: Replica: 97fc6445d9244db78dd6284e19b22409, State: Running, Role: LEADER
I20260811 18:44:04.728077 27129 consensus_queue.cc:237] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97fc6445d9244db78dd6284e19b22409" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33119 } } peers { permanent_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37699 } } peers { permanent_uuid: "21050d666d634b858289431412d37063" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34335 } }
I20260811 18:44:04.732854 26266 catalog_manager.cc:5495] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97fc6445d9244db78dd6284e19b22409 (127.30.185.130). New cstate: current_term: 1 leader_uuid: "97fc6445d9244db78dd6284e19b22409" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97fc6445d9244db78dd6284e19b22409" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 33119 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21050d666d634b858289431412d37063" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34335 } health_report { overall_health: UNKNOWN } } }
I20260811 18:44:04.760013 27109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:04.754291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58378 (local address 127.30.185.130:33119)
0811 18:44:04.754515 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:04.754521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:04.754542 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:44:04.754765 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:04.754769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:04.754846 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:04.755017 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:44:04.755392 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.756135 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.757781 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.758209 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.758349 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:04.758875 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:44:04.759402 (+ 527us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:44:04.759580 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:04.759634 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 18:44:04.759717 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 18:44:04.768868 27007 raft_consensus.cc:1261] T ed54559ac5d94f6d987221ab32083a04 P 21050d666d634b858289431412d37063 [term 1 FOLLOWER]: Refusing update from remote peer 97fc6445d9244db78dd6284e19b22409: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:44:04.768853 26525 raft_consensus.cc:1261] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045 [term 1 FOLLOWER]: Refusing update from remote peer 97fc6445d9244db78dd6284e19b22409: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:44:04.770020 27136 consensus_queue.cc:1035] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9814a8e125c4eeb8a479d67ef8b2045" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 37699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:44:04.770380 27129 consensus_queue.cc:1035] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21050d666d634b858289431412d37063" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:44:04.782313 27140 mvcc.cc:205] Tried to move back new op lower bound from 7317396868162240512 to 7317396868010446848. Current Snapshot: MvccSnapshot[applied={T|T < 7317396868162240512}]
I20260811 18:44:04.783959 27143 mvcc.cc:205] Tried to move back new op lower bound from 7317396868162240512 to 7317396868010446848. Current Snapshot: MvccSnapshot[applied={T|T < 7317396868162240512}]
I20260811 18:44:04.797418 26263 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:47214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:44:04.797745 26263 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:47214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:44:04.801704 26263 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b: Sending DeleteTablet for 3 replicas of tablet ed54559ac5d94f6d987221ab32083a04
I20260811 18:44:04.803112 26729 tablet_service.cc:1514] Processing DeleteTablet for tablet ed54559ac5d94f6d987221ab32083a04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:44:04 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:58364
I20260811 18:44:04.803139 26969 tablet_service.cc:1514] Processing DeleteTablet for tablet ed54559ac5d94f6d987221ab32083a04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:44:04 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:48164
I20260811 18:44:04.803941 26495 tablet_service.cc:1514] Processing DeleteTablet for tablet ed54559ac5d94f6d987221ab32083a04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:44:04 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:47892
I20260811 18:44:04.804208 27159 tablet_replica.cc:330] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409: stopping tablet replica
I20260811 18:44:04.804760 27159 raft_consensus.cc:2229] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:44:04.805140 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 26349
I20260811 18:44:04.805670 27159 raft_consensus.cc:2258] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:44:04.807734 27161 tablet_replica.cc:330] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045: stopping tablet replica
I20260811 18:44:04.808164 27161 raft_consensus.cc:2229] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:44:04.808439 27159 ts_tablet_manager.cc:1905] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:44:04.808697 27161 raft_consensus.cc:2258] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:44:04.813603 27161 ts_tablet_manager.cc:1905] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:44:04.816628 27159 ts_tablet_manager.cc:1918] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:44:04.816910 27159 log.cc:1196] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/ed54559ac5d94f6d987221ab32083a04
I20260811 18:44:04.817726 27159 ts_tablet_manager.cc:1939] T ed54559ac5d94f6d987221ab32083a04 P 97fc6445d9244db78dd6284e19b22409: Deleting consensus metadata
I20260811 18:44:04.819871 26241 catalog_manager.cc:4841] TS 97fc6445d9244db78dd6284e19b22409 (127.30.185.130:33119): tablet ed54559ac5d94f6d987221ab32083a04 (table test-table [id=7553ce26cbf24a678c25ce905265aad8]) successfully deleted
I20260811 18:44:04.824105 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:04.806700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47900 (local address 127.30.185.129:37699)
0811 18:44:04.806893 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:04.806899 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:04.806919 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:44:04.806980 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:04.806986 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:04.807071 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:04.807187 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:04.809260 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.810435 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.812332 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:04.812997 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:04.813151 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:04.816816 (+ 3665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:04.816833 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:04.816863 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:04.816911 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:04.820812 (+ 3901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:04.821785 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:04.821796 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:04.821799 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:04.821975 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:04.822436 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:04.822441 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:04.822444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:04.822816 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:04.823168 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:04.823458 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 18:44:04.823689 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 18:44:04.824910 26596 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:44:04.825340 27160 tablet_replica.cc:330] T ed54559ac5d94f6d987221ab32083a04 P 21050d666d634b858289431412d37063: stopping tablet replica
I20260811 18:44:04.826098 27160 raft_consensus.cc:2229] T ed54559ac5d94f6d987221ab32083a04 P 21050d666d634b858289431412d37063 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:44:04.826799 27160 raft_consensus.cc:2258] T ed54559ac5d94f6d987221ab32083a04 P 21050d666d634b858289431412d37063 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:44:04.829537 27160 ts_tablet_manager.cc:1905] T ed54559ac5d94f6d987221ab32083a04 P 21050d666d634b858289431412d37063: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:44:04.834792 27160 ts_tablet_manager.cc:1918] T ed54559ac5d94f6d987221ab32083a04 P 21050d666d634b858289431412d37063: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:44:04.835004 27160 log.cc:1196] T ed54559ac5d94f6d987221ab32083a04 P 21050d666d634b858289431412d37063: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/ed54559ac5d94f6d987221ab32083a04
I20260811 18:44:04.835530 27160 ts_tablet_manager.cc:1939] T ed54559ac5d94f6d987221ab32083a04 P 21050d666d634b858289431412d37063: Deleting consensus metadata
I20260811 18:44:04.837272 26245 catalog_manager.cc:4841] TS 21050d666d634b858289431412d37063 (127.30.185.131:34335): tablet ed54559ac5d94f6d987221ab32083a04 (table test-table [id=7553ce26cbf24a678c25ce905265aad8]) successfully deleted
I20260811 18:44:04.973827 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 26349
W20260811 18:44:05.005251 26241 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.185.129:37699 (error 108)
I20260811 18:44:05.005465 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 26616
I20260811 18:44:05.007294 27114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:05.006820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:37699 (local address 127.0.0.1:47904)
0811 18:44:05.006987 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:05.007112 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 18:44:05.008670 26241 catalog_manager.cc:4584] TS a9814a8e125c4eeb8a479d67ef8b2045 (127.30.185.129:37699): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ed54559ac5d94f6d987221ab32083a04: Network error: Client connection negotiation failed: client connection to 127.30.185.129:37699: connect: Connection refused (error 111)
I20260811 18:44:05.022871 27109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:05.006423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58384 (local address 127.30.185.130:33119)
0811 18:44:05.006627 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:05.006639 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:05.006661 (+ 22us) server_negotiation.cc:407] Connection header received
0811 18:44:05.006846 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:05.006852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:05.006938 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:05.007052 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:05.008592 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:05.009936 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:05.011292 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:05.012008 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:05.012142 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:05.015169 (+ 3027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:05.015190 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:05.015212 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:05.015264 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:05.019650 (+ 4386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:05.020451 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:05.020467 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:05.020472 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:05.020647 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:05.021056 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:05.021061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:05.021065 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:05.021576 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:05.021934 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:05.022219 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 18:44:05.022455 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 18:44:05.023721 26849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:44:05.062348 27114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:05.061877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:37699 (local address 127.0.0.1:47912)
0811 18:44:05.062089 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:05.062198 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 18:44:05.063433 27114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:05.063076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:37699 (local address 127.0.0.1:47920)
0811 18:44:05.063224 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:05.063303 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 18:44:05.124270 27114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:05.123693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:37699 (local address 127.0.0.1:47932)
0811 18:44:05.123976 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:05.124079 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260811 18:44:05.125509 27114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:05.125083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:37699 (local address 127.0.0.1:47934)
0811 18:44:05.125272 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:05.125349 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 18:44:05.142442 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 26616
I20260811 18:44:05.173619 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 26863
I20260811 18:44:05.188077 27115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:05.174438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48168 (local address 127.30.185.131:34335)
0811 18:44:05.174616 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:05.174621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:05.174696 (+ 75us) server_negotiation.cc:407] Connection header received
0811 18:44:05.174873 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:05.174877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:05.174943 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:05.175031 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:05.176655 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:05.177459 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:05.178785 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:05.179182 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:05.179278 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:05.182322 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:05.182338 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:05.182357 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:05.182398 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:05.185533 (+ 3135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:05.186262 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:05.186269 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:05.186272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:05.186426 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:05.186772 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:05.186775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:05.186777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:05.187122 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:05.187347 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:05.187613 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 18:44:05.187782 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:44:05.188838 27084 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:44:05.206785 27114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:05.206281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:37699 (local address 127.0.0.1:47940)
0811 18:44:05.206496 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:05.206594 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 18:44:05.207917 27114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:05.207525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:37699 (local address 127.0.0.1:47956)
0811 18:44:05.207727 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:05.207801 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 18:44:05.282215 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 26863
I20260811 18:44:05.306931 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 26213
I20260811 18:44:05.308240 26337 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:44:05.425405 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 26213
I20260811 18:44:05.425462 27114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:05.424875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:37699 (local address 127.0.0.1:47960)
0811 18:44:05.425110 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:05.425269 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 18:44:05.458360 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:46035
--webserver_interface=127.30.185.190
--webserver_port=43895
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:39755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:46035
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:44:05.831265 27171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:44:05.831581 27171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:44:05.831756 27171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:44:05.839234 27171 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:44:05.839363 27171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:44:05.839458 27171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:44:05.839573 27171 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:44:05.839653 27171 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:44:05.849977 27171 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:39755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:46035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:46035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=43895
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:44:05.851430 27171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:44:05.852941 27171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:44:05.865512 27176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:05.865610 27177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:05.865568 27179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:44:05.866580 27171 server_base.cc:1034] running on GCE node
I20260811 18:44:05.867471 27171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:44:05.868964 27171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:44:05.870203 27171 hybrid_clock.cc:648] HybridClock initialized: now 1786473845870158 us; error 58 us; skew 500 ppm
I20260811 18:44:05.870597 27171 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:44:05 dist-test-slave-z8x9 krb5kdc[25226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473845, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:44:05.873951 27171 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:44:05.875412 27171 webserver.cc:460] Webserver started at http://127.30.185.190:43895/ using document root <none> and password file <none>
I20260811 18:44:05.876055 27171 fs_manager.cc:359] Metadata directory not provided
I20260811 18:44:05.876160 27171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:44:05.881732 27171 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:44:05.884552 27187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:05.885926 27171 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:44:05.886070 27171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "0d1f7192ff6e4019b62175cfae0a551b"
format_stamp: "Formatted at 2026-08-11 18:43:59 on dist-test-slave-z8x9"
server_key: "bb7590aea56c12beca5a0b2dfaf0c441"
server_key_iv: "317c63af76dcc94e7eca65740af8bb42"
server_key_version: "encryptionkey@0"
I20260811 18:44:05.886986 27171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:44:05.905031 27171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:44:05.910493 27171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:44:05.910849 27171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:44:05.931465 27171 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:46035
I20260811 18:44:05.931473 27299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:46035 every 8 connection(s)
I20260811 18:44:05.933032 27171 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:44:05.940110 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 27171
I20260811 18:44:05.940291 27301 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:44:05.941248 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:37699
--local_ip_for_outbound_sockets=127.30.185.129
--tserver_master_addrs=127.30.185.190:46035
--webserver_port=44951
--webserver_interface=127.30.185.129
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:39755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:44:05.945550 27301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b: Bootstrap starting.
I20260811 18:44:05.969457 27301 log.cc:824] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:44:05.986940 27301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:44:05.987529 27301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b: Bootstrap complete.
I20260811 18:44:05.997550 27301 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1f7192ff6e4019b62175cfae0a551b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46035 } }
I20260811 18:44:05.998485 27301 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d1f7192ff6e4019b62175cfae0a551b, State: Initialized, Role: FOLLOWER
I20260811 18:44:05.999157 27301 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1f7192ff6e4019b62175cfae0a551b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46035 } }
I20260811 18:44:05.999411 27301 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:44:05.999539 27301 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:44:05.999740 27301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 2 FOLLOWER]: Advancing to term 3
I20260811 18:44:06.003790 27301 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1f7192ff6e4019b62175cfae0a551b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46035 } }
I20260811 18:44:06.004362 27301 leader_election.cc:302] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d1f7192ff6e4019b62175cfae0a551b; no voters:
I20260811 18:44:06.005139 27301 leader_election.cc:288] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 18:44:06.005332 27306 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 3 FOLLOWER]: Leader election won for term 3
I20260811 18:44:06.007756 27301 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:44:06.009526 27306 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [term 3 LEADER]: Becoming Leader. State: Replica: 0d1f7192ff6e4019b62175cfae0a551b, State: Running, Role: LEADER
I20260811 18:44:06.010311 27306 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1f7192ff6e4019b62175cfae0a551b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46035 } }
I20260811 18:44:06.013415 27308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d1f7192ff6e4019b62175cfae0a551b. Latest consensus state: current_term: 3 leader_uuid: "0d1f7192ff6e4019b62175cfae0a551b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1f7192ff6e4019b62175cfae0a551b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46035 } } }
I20260811 18:44:06.013882 27308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [sys.catalog]: This master's current role is: LEADER
I20260811 18:44:06.013787 27307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0d1f7192ff6e4019b62175cfae0a551b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1f7192ff6e4019b62175cfae0a551b" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46035 } } }
I20260811 18:44:06.014103 27307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d1f7192ff6e4019b62175cfae0a551b [sys.catalog]: This master's current role is: LEADER
I20260811 18:44:06.014845 27317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:44:06.037405 27317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:44:06.039525 27317 catalog_manager.cc:1259] Loaded cluster ID: bb74d411909049e09385a3e96c736e18
I20260811 18:44:06.039690 27317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 18:44:06.042645 27317 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7faefb0289ad google::LogMessage::Fail() at ??:0
@ 0x7faefb02b88a google::LogMessage::SendToLog() at ??:0
@ 0x7faefb0284de google::LogMessage::Flush() at ??:0
@ 0x7faefb029a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7faf10a5b512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7faf10a58976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7faf10ae0610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7faf10ae0439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7faefdc74345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7faefdc7791c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7faefdc77799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7faefdc52eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7faf00f2c6db start_thread at ??:0
@ 0x7faef9ec261f clone at ??:0
W20260811 18:44:06.340508 27304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:44:06.340862 27304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:44:06.341035 27304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:44:06.348829 27304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:44:06.349007 27304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:44:06.349238 27304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:44:06.359823 27304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:39755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:37699
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=44951
--tserver_master_addrs=127.30.185.190:46035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:44:06.361286 27304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:44:06.362656 27304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:44:06.376180 27369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:06.376133 27372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:06.376199 27374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:44:06.377116 27304 server_base.cc:1034] running on GCE node
I20260811 18:44:06.377817 27304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:44:06.378813 27304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:44:06.380056 27304 hybrid_clock.cc:648] HybridClock initialized: now 1786473846380018 us; error 62 us; skew 500 ppm
I20260811 18:44:06.380407 27304 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:44:06 dist-test-slave-z8x9 krb5kdc[25226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473846, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:44:06.383922 27304 init.cc:377] Logged in from keytab as kudu/127.30.185.129@KRBTEST.COM (short username kudu)
I20260811 18:44:06.385639 27304 webserver.cc:460] Webserver started at http://127.30.185.129:44951/ using document root <none> and password file <none>
I20260811 18:44:06.386376 27304 fs_manager.cc:359] Metadata directory not provided
I20260811 18:44:06.386536 27304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:44:06.392217 27304 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.006s
I20260811 18:44:06.395757 27382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:06.397322 27304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 18:44:06.397507 27304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "a9814a8e125c4eeb8a479d67ef8b2045"
format_stamp: "Formatted at 2026-08-11 18:44:00 on dist-test-slave-z8x9"
server_key: "bcc3bb38a8b883c752bcced5e84fdab6"
server_key_iv: "692178b074c55d6786d25c63b6a9f336"
server_key_version: "encryptionkey@0"
I20260811 18:44:06.398532 27304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:44:06.418227 27304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:44:06.422808 27304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:44:06.423111 27304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:44:06.424170 27304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:44:06.426190 27390 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260811 18:44:06.431936 27390 ts_tablet_manager.cc:1852] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260811 18:44:06.432963 27390 ts_tablet_manager.cc:1905] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:44:06.440696 27390 ts_tablet_manager.cc:1918] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:44:06.440908 27390 log.cc:1196] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/wal/wals/ed54559ac5d94f6d987221ab32083a04
I20260811 18:44:06.441385 27390 ts_tablet_manager.cc:1939] T ed54559ac5d94f6d987221ab32083a04 P a9814a8e125c4eeb8a479d67ef8b2045: Deleting consensus metadata
I20260811 18:44:06.441774 27304 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260811 18:44:06.441933 27304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.016s user 0.001s sys 0.000s
I20260811 18:44:06.442091 27304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:44:06.442225 27304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:06.479059 27304 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:37699
I20260811 18:44:06.479110 27565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:37699 every 8 connection(s)
I20260811 18:44:06.480525 27304 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:44:06.485698 27568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:06.483667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.129:35969)
0811 18:44:06.484440 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:06.484722 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":319,"threads_started":1}
W20260811 18:44:06.486374 27566 heartbeater.cc:644] Failed to heartbeat to 127.30.185.190:46035 (0 consecutive failures): Network error: Failed to ping master at 127.30.185.190:46035: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
I20260811 18:44:06.487238 27568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:06.486905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.129:59895)
0811 18:44:06.487055 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:06.487127 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 18:44:06.488062 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 27304
I20260811 18:44:06.488276 27568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:06.487892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.129:55035)
0811 18:44:06.488025 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:06.488099 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 18:44:06.488593 27566 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 18:44:06.488868 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:33119
--local_ip_for_outbound_sockets=127.30.185.130
--tserver_master_addrs=127.30.185.190:46035
--webserver_port=43679
--webserver_interface=127.30.185.130
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:39755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:44:06.841830 27569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:44:06.842145 27569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:44:06.842370 27569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:44:06.850260 27569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:44:06.850394 27569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:44:06.850579 27569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:44:06.861050 27569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:39755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:33119
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=43679
--tserver_master_addrs=127.30.185.190:46035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:44:06.862583 27569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:44:06.863977 27569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:44:06.878665 27583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:06.878782 27580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:06.878806 27577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:44:06.880440 27569 server_base.cc:1034] running on GCE node
I20260811 18:44:06.881384 27569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:44:06.882475 27569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:44:06.883726 27569 hybrid_clock.cc:648] HybridClock initialized: now 1786473846883667 us; error 51 us; skew 500 ppm
I20260811 18:44:06.884145 27569 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:44:06 dist-test-slave-z8x9 krb5kdc[25226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473846, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:44:06.888206 27569 init.cc:377] Logged in from keytab as kudu/127.30.185.130@KRBTEST.COM (short username kudu)
I20260811 18:44:06.889882 27569 webserver.cc:460] Webserver started at http://127.30.185.130:43679/ using document root <none> and password file <none>
I20260811 18:44:06.890592 27569 fs_manager.cc:359] Metadata directory not provided
I20260811 18:44:06.890709 27569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:44:06.896612 27569 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 18:44:06.899834 27590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:06.901652 27569 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:44:06.901876 27569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "97fc6445d9244db78dd6284e19b22409"
format_stamp: "Formatted at 2026-08-11 18:44:00 on dist-test-slave-z8x9"
server_key: "b74b4025ff34bc4a1f221c5b346f0227"
server_key_iv: "48f52eccb0e58911547dcfa730d986f3"
server_key_version: "encryptionkey@0"
I20260811 18:44:06.903057 27569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:44:06.919560 27569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:44:06.924650 27569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:44:06.924997 27569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:44:06.926211 27569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:44:06.927752 27569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:44:06.927870 27569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:06.928032 27569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:44:06.928121 27569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:06.965922 27569 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:33119
I20260811 18:44:06.966046 27795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:33119 every 8 connection(s)
I20260811 18:44:06.967545 27569 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:44:06.971333 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 27569
I20260811 18:44:06.972352 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:34335
--local_ip_for_outbound_sockets=127.30.185.131
--tserver_master_addrs=127.30.185.190:46035
--webserver_port=40431
--webserver_interface=127.30.185.131
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:39755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:44:06.973451 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:06.970763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.130:50315)
0811 18:44:06.971771 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:06.972177 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":403,"threads_started":1}
W20260811 18:44:06.974197 27796 heartbeater.cc:644] Failed to heartbeat to 127.30.185.190:46035 (0 consecutive failures): Network error: Failed to ping master at 127.30.185.190:46035: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
I20260811 18:44:06.975204 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:06.974859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.130:36751)
0811 18:44:06.975012 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:06.975094 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 18:44:06.976270 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:06.975936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.130:53043)
0811 18:44:06.976085 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:06.976162 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 18:44:06.976603 27796 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:44:07.318869 27803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:44:07.319166 27803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:44:07.319267 27803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:44:07.326424 27803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:44:07.326538 27803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:44:07.326651 27803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:44:07.336251 27803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:39755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:34335
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=40431
--tserver_master_addrs=127.30.185.190:46035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:44:07.337579 27803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:44:07.338862 27803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:44:07.352515 27810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:07.352558 27811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:07.352824 27813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:44:07.353602 27803 server_base.cc:1034] running on GCE node
I20260811 18:44:07.354326 27803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:44:07.355365 27803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:44:07.356613 27803 hybrid_clock.cc:648] HybridClock initialized: now 1786473847356556 us; error 53 us; skew 500 ppm
I20260811 18:44:07.356958 27803 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:44:07 dist-test-slave-z8x9 krb5kdc[25226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473847, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:44:07.360443 27803 init.cc:377] Logged in from keytab as kudu/127.30.185.131@KRBTEST.COM (short username kudu)
I20260811 18:44:07.362080 27803 webserver.cc:460] Webserver started at http://127.30.185.131:40431/ using document root <none> and password file <none>
I20260811 18:44:07.363051 27803 fs_manager.cc:359] Metadata directory not provided
I20260811 18:44:07.363222 27803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:44:07.368742 27803 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 18:44:07.371585 27825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:07.373030 27803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 18:44:07.373281 27803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "21050d666d634b858289431412d37063"
format_stamp: "Formatted at 2026-08-11 18:44:01 on dist-test-slave-z8x9"
server_key: "99c07aea91736332afe04d955b569409"
server_key_iv: "05e37c42b36a428bd37d3dc2190a60d9"
server_key_version: "encryptionkey@0"
I20260811 18:44:07.374318 27803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:44:07.393018 27803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:44:07.397289 27803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:44:07.397600 27803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:44:07.398692 27803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:44:07.400214 27803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:44:07.400349 27803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:07.400485 27803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:44:07.400596 27803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:07.436079 27803 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:34335
I20260811 18:44:07.436179 28022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:34335 every 8 connection(s)
I20260811 18:44:07.437695 27803 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:44:07.442526 28026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:07.440641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.131:54647)
0811 18:44:07.441443 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:07.441659 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":315,"threads_started":1}
W20260811 18:44:07.443313 28024 heartbeater.cc:644] Failed to heartbeat to 127.30.185.190:46035 (0 consecutive failures): Network error: Failed to ping master at 127.30.185.190:46035: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
I20260811 18:44:07.444092 28026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:07.443785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.131:33969)
0811 18:44:07.443923 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:07.443991 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 18:44:07.444974 28026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:07.444682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.131:58079)
0811 18:44:07.444806 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:07.444876 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 18:44:07.445034 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 27803
W20260811 18:44:07.445305 28024 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 18:44:07.490088 28030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:07.489290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.129:60463)
0811 18:44:07.489762 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:07.489859 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":223,"threads_started":1}
I20260811 18:44:07.978101 28033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:07.977150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.130:54743)
0811 18:44:07.977670 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:07.977813 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":206,"threads_started":1}
I20260811 18:44:08.446622 28034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:08.445815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.131:50361)
0811 18:44:08.446266 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:08.446398 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":218,"threads_started":1}
I20260811 18:44:08.491737 28035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:08.491053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.129:60157)
0811 18:44:08.491492 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:08.491574 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":246,"threads_started":1}
I20260811 18:44:08.979809 28037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:08.979071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.130:51871)
0811 18:44:08.979549 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:08.979636 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":223,"threads_started":1}
I20260811 18:44:09.448320 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:09.447540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.131:46591)
0811 18:44:09.448052 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:09.448138 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":265,"threads_started":1}
I20260811 18:44:09.493659 28041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:09.492735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.129:54727)
0811 18:44:09.493349 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:09.493460 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":269,"threads_started":1}
I20260811 18:44:09.981705 28042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:09.980832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.130:50863)
0811 18:44:09.981424 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:09.981520 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":271,"threads_started":1}
I20260811 18:44:10.450075 28043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:10.449328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.131:56525)
0811 18:44:10.449814 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:10.449899 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":259,"threads_started":1}
I20260811 18:44:10.495396 28047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:10.494610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.129:42805)
0811 18:44:10.495088 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:10.495218 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":248,"threads_started":1}
I20260811 18:44:10.983552 28048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:10.982765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.130:46023)
0811 18:44:10.983265 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:10.983371 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":233,"threads_started":1}
I20260811 18:44:11.451869 28049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:11.451041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.131:45985)
0811 18:44:11.451577 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:11.451666 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":300,"threads_started":1}
I20260811 18:44:11.497325 28056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:11.496425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.129:44867)
0811 18:44:11.496953 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:11.497044 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":289,"threads_started":1}
I20260811 18:44:11.985332 28058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:11.984500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.130:36713)
0811 18:44:11.984984 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:11.985071 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":235,"threads_started":1}
I20260811 18:44:12.453816 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:12.453010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.131:50495)
0811 18:44:12.453554 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:12.453639 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":302,"threads_started":1}
I20260811 18:44:12.499004 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:12.498301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.129:55229)
0811 18:44:12.498752 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:12.498840 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":242,"threads_started":1}
I20260811 18:44:12.986940 28063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:12.986173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.130:38003)
0811 18:44:12.986670 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:12.986767 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":249,"threads_started":1}
I20260811 18:44:13.455495 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:13.454730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.131:33501)
0811 18:44:13.455217 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:13.455326 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":248,"threads_started":1}
I20260811 18:44:13.500818 28065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:13.499953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.129:43565)
0811 18:44:13.500510 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:13.500602 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":264,"threads_started":1}
I20260811 18:44:13.988900 28067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:13.988005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.130:34675)
0811 18:44:13.988606 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:13.988685 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":270,"threads_started":1}
I20260811 18:44:14.457674 28068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:14.456558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.131:37795)
0811 18:44:14.457237 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:14.457387 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":282,"threads_started":1}
I20260811 18:44:14.502681 28070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:14.501861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.129:53451)
0811 18:44:14.502336 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:14.502434 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":261,"threads_started":1}
I20260811 18:44:14.990598 28073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:14.989885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.130:34003)
0811 18:44:14.990331 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:14.990424 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":221,"threads_started":1}
I20260811 18:44:15.459403 28074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:15.458661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.131:53097)
0811 18:44:15.459125 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:15.459213 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":251,"threads_started":1}
I20260811 18:44:15.504436 28075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:15.503625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.129:47759)
0811 18:44:15.504103 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:15.504245 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":245,"threads_started":1}
I20260811 18:44:15.992399 28076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:15.991548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.130:60075)
0811 18:44:15.992080 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:15.992210 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":259,"threads_started":1}
I20260811 18:44:16.461390 28077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:16.460453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.131:36857)
0811 18:44:16.461009 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:16.461104 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":273,"threads_started":1}
I20260811 18:44:16.506230 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:16.505431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.129:52169)
0811 18:44:16.505956 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:16.506046 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":299,"threads_started":1}
I20260811 18:44:16.994304 28079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:16.993549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.130:45441)
0811 18:44:16.994023 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:16.994111 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":253,"threads_started":1}
I20260811 18:44:17.463271 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:17.462392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.131:39477)
0811 18:44:17.462985 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:17.463078 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":346,"threads_started":1}
I20260811 18:44:17.508167 28082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:17.507255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.129:50927)
0811 18:44:17.507802 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:17.507895 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":317,"threads_started":1}
I20260811 18:44:17.996168 28084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:17.995346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.130:42945)
0811 18:44:17.995895 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:17.995986 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":309,"threads_started":1}
I20260811 18:44:18.465229 28087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:18.464318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.131:35061)
0811 18:44:18.464879 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:18.464981 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":333,"threads_started":1}
I20260811 18:44:18.509912 28088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:18.509103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.129:54517)
0811 18:44:18.509620 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:18.509709 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":303,"threads_started":1}
I20260811 18:44:19.000494 28092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:18.997209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.130:38605)
0811 18:44:19.000162 (+ 2953us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:19.000278 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2682,"thread_start_us":2563,"threads_started":1}
I20260811 18:44:19.467162 28094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:19.466339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.131:38915)
0811 18:44:19.466899 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:19.466991 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":305,"threads_started":1}
I20260811 18:44:19.511925 28096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:19.510955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.129:41559)
0811 18:44:19.511497 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:19.511613 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":254,"threads_started":1}
I20260811 18:44:20.003249 28097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:20.001860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.130:33993)
0811 18:44:20.002921 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:20.003045 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":877,"thread_start_us":686,"threads_started":1}
I20260811 18:44:20.469295 28098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:20.468255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.131:50367)
0811 18:44:20.468876 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:20.468995 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":308,"threads_started":1}
I20260811 18:44:20.513864 28101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:20.512999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.129:49025)
0811 18:44:20.513575 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:20.513667 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":319,"threads_started":1}
I20260811 18:44:21.005286 28102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:21.004437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.130:53651)
0811 18:44:21.004920 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:21.005009 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":241,"threads_started":1}
I20260811 18:44:21.470913 28104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:21.470223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.131:45015)
0811 18:44:21.470645 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:21.470724 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":218,"threads_started":1}
I20260811 18:44:21.515894 28105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:21.515106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.129:51247)
0811 18:44:21.515593 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:21.515694 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":253,"threads_started":1}
I20260811 18:44:22.007421 28107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:22.006524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.130:41919)
0811 18:44:22.007019 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:22.007122 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":267,"threads_started":1}
I20260811 18:44:22.447715 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 27304
I20260811 18:44:22.473250 28113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:22.472199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46035 (local address 127.30.185.131:58497)
0811 18:44:22.472834 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:22.472973 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:46035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":314,"threads_started":1}
I20260811 18:44:22.474080 28109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:22.449238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49504 (local address 127.30.185.129:37699)
0811 18:44:22.450263 (+ 1025us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:22.450277 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:22.450321 (+ 44us) server_negotiation.cc:407] Connection header received
0811 18:44:22.452169 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:22.452180 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:22.452641 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:22.452971 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:22.454640 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:22.456001 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:22.457438 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:22.458176 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:22.458333 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:22.465834 (+ 7501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:22.465846 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:22.465866 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:22.465918 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:22.469583 (+ 3665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:22.470686 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:22.470695 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:22.470703 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:22.470914 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:22.471455 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:22.471458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:22.471460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:22.472296 (+ 836us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:22.472693 (+ 397us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:22.473252 (+ 559us) server_negotiation.cc:299] Negotiation successful
0811 18:44:22.473799 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":805,"thread_start_us":676,"threads_started":1}
I20260811 18:44:22.475644 27556 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:44:22.567808 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 27304
I20260811 18:44:22.594228 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 27569
I20260811 18:44:22.614073 28123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:22.595398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45704 (local address 127.30.185.130:33119)
0811 18:44:22.595970 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:22.595986 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:22.596026 (+ 40us) server_negotiation.cc:407] Connection header received
0811 18:44:22.597726 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:22.597738 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:22.598206 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:22.598518 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:22.600002 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:22.601548 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:22.602973 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:22.603713 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:22.603890 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:22.606851 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:22.606896 (+ 45us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:22.606924 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:22.606988 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:22.610464 (+ 3476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:22.611160 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:22.611168 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:22.611176 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:22.611348 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:22.611754 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:22.611782 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:22.611785 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:22.612417 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:22.612681 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:22.613119 (+ 438us) server_negotiation.cc:299] Negotiation successful
0811 18:44:22.613762 (+ 643us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":281,"threads_started":1}
I20260811 18:44:22.615585 27791 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:44:22.708077 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 27569
I20260811 18:44:22.738828 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 27803
I20260811 18:44:22.758774 28125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:22.740176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53314 (local address 127.30.185.131:34335)
0811 18:44:22.740850 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:22.740864 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:22.740908 (+ 44us) server_negotiation.cc:407] Connection header received
0811 18:44:22.742643 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:22.742653 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:22.743097 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:22.743406 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:22.745139 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:22.746456 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:22.747972 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:22.748665 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:22.748806 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:22.751991 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:22.751999 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:22.752016 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:22.752055 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:22.755291 (+ 3236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:22.756066 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:22.756073 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:22.756079 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:22.756251 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:22.756643 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:22.756646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:22.756648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:22.757319 (+ 671us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:22.757566 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:22.757967 (+ 401us) server_negotiation.cc:299] Negotiation successful
0811 18:44:22.758457 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":337,"threads_started":1}
I20260811 18:44:22.760245 28017 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:44:22.847093 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 27803
2026-08-11T18:44:22Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25316 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:44:22 dist-test-slave-z8x9 krb5kdc[28129](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:44:22 dist-test-slave-z8x9 krb5kdc[28129](info): set up 2 sockets
Aug 11 18:44:22 dist-test-slave-z8x9 krb5kdc[28129](info): commencing operation
krb5kdc: starting...
W20260811 18:44:24.299009 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.399s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:44:24 dist-test-slave-z8x9 krb5kdc[28129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:44:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:44:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:44:24.638135 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:36929
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:36929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:44:24.994897 28153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:44:24.995234 28153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:44:24.995395 28153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:44:25.002887 28153 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 18:44:25.003002 28153 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 18:44:25.003086 28153 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 18:44:25.003177 28153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:44:25.003279 28153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:44:25.003417 28153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:44:25.003496 28153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:44:25.003587 28153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:44:25.013893 28153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:36929
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:36929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:44:25.015298 28153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:44:25.016780 28153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:44:25.029824 28166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:25.029860 28160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:25.029785 28163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:44:25.030555 28153 server_base.cc:1034] running on GCE node
I20260811 18:44:25.031518 28153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:44:25.033025 28153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:44:25.034298 28153 hybrid_clock.cc:648] HybridClock initialized: now 1786473865034230 us; error 82 us; skew 500 ppm
I20260811 18:44:25.034606 28153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:44:25 dist-test-slave-z8x9 krb5kdc[28129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473865, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:44:25.038134 28153 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:44:25.039753 28153 webserver.cc:460] Webserver started at http://127.30.185.190:42741/ using document root <none> and password file <none>
I20260811 18:44:25.040648 28153 fs_manager.cc:359] Metadata directory not provided
I20260811 18:44:25.040800 28153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:44:25.041139 28153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:44:25.044407 28153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "5cd1365a8a1047d1bec51d03529ad212"
format_stamp: "Formatted at 2026-08-11 18:44:25 on dist-test-slave-z8x9"
server_key: "4a5725a31e5b88d8e5f411e7dc921749"
server_key_iv: "faa9a9d0311eef54f135095635648f82"
server_key_version: "encryptionkey@0"
I20260811 18:44:25.045325 28153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "5cd1365a8a1047d1bec51d03529ad212"
format_stamp: "Formatted at 2026-08-11 18:44:25 on dist-test-slave-z8x9"
server_key: "4a5725a31e5b88d8e5f411e7dc921749"
server_key_iv: "faa9a9d0311eef54f135095635648f82"
server_key_version: "encryptionkey@0"
I20260811 18:44:25.050865 28153 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 18:44:25.054538 28182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:25.055915 28153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:44:25.056128 28153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "5cd1365a8a1047d1bec51d03529ad212"
format_stamp: "Formatted at 2026-08-11 18:44:25 on dist-test-slave-z8x9"
server_key: "4a5725a31e5b88d8e5f411e7dc921749"
server_key_iv: "faa9a9d0311eef54f135095635648f82"
server_key_version: "encryptionkey@0"
I20260811 18:44:25.056378 28153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:44:25.072160 28153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:44:25.076527 28153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:44:25.076839 28153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:44:25.097371 28153 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:36929
I20260811 18:44:25.097427 28276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:36929 every 8 connection(s)
I20260811 18:44:25.098884 28153 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:44:25.101833 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 28153
I20260811 18:44:25.102077 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:44:25.102792 31462 external_mini_cluster.cc:1428] Setting key 607d0f893471a2f2cfde3bcdf6b83d63
I20260811 18:44:25.103062 28277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:44:25.118104 28277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212: Bootstrap starting.
Aug 11 18:44:25 dist-test-slave-z8x9 krb5kdc[28129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:44:25.122131 28277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212: Neither blocks nor log segments found. Creating new log.
I20260811 18:44:25.123620 28277 log.cc:824] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212: Log is configured to *not* fsync() on all Append() calls
I20260811 18:44:25.127351 28277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212: No bootstrap required, opened a new log
I20260811 18:44:25.129992 28283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:25.105019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36660 (local address 127.30.185.190:36929)
0811 18:44:25.105928 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:25.105948 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:25.106003 (+ 55us) server_negotiation.cc:407] Connection header received
0811 18:44:25.108258 (+ 2255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:25.108287 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:25.108819 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:25.109342 (+ 523us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:25.110798 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:25.112765 (+ 1967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:25.114331 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:25.115473 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:25.115672 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:25.120489 (+ 4817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:25.120500 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:25.120518 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:25.120569 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:25.124705 (+ 4136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:25.125687 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:25.125696 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:25.125704 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:25.125922 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:25.126348 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:25.126353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:25.126356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:25.127108 (+ 752us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:25.127436 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:25.128034 (+ 598us) server_negotiation.cc:299] Negotiation successful
0811 18:44:25.128566 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":358,"threads_started":1}
I20260811 18:44:25.136941 28277 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd1365a8a1047d1bec51d03529ad212" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 36929 } }
I20260811 18:44:25.137460 28277 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:44:25.137550 28277 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd1365a8a1047d1bec51d03529ad212, State: Initialized, Role: FOLLOWER
I20260811 18:44:25.138142 28277 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd1365a8a1047d1bec51d03529ad212" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 36929 } }
I20260811 18:44:25.138381 28277 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:44:25.138491 28277 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:44:25.138630 28277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:44:25.140846 28277 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd1365a8a1047d1bec51d03529ad212" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 36929 } }
I20260811 18:44:25.141429 28277 leader_election.cc:302] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cd1365a8a1047d1bec51d03529ad212; no voters:
I20260811 18:44:25.142033 28277 leader_election.cc:288] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:44:25.142261 28285 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:44:25.143679 28285 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 1 LEADER]: Becoming Leader. State: Replica: 5cd1365a8a1047d1bec51d03529ad212, State: Running, Role: LEADER
I20260811 18:44:25.144270 28285 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd1365a8a1047d1bec51d03529ad212" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 36929 } }
I20260811 18:44:25.144474 28277 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:44:25.147411 28286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cd1365a8a1047d1bec51d03529ad212" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd1365a8a1047d1bec51d03529ad212" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 36929 } } }
I20260811 18:44:25.147358 28289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cd1365a8a1047d1bec51d03529ad212. Latest consensus state: current_term: 1 leader_uuid: "5cd1365a8a1047d1bec51d03529ad212" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd1365a8a1047d1bec51d03529ad212" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 36929 } } }
I20260811 18:44:25.154238 28289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [sys.catalog]: This master's current role is: LEADER
I20260811 18:44:25.155292 28286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [sys.catalog]: This master's current role is: LEADER
I20260811 18:44:25.156912 28303 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:44:25.162915 28303 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:44:25.172873 28303 catalog_manager.cc:1347] Generated new cluster ID: e59b1719777b4493b26dbd931f8edc00
I20260811 18:44:25.173024 28303 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:44:25.191335 28303 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:44:25.193120 28303 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:44:25.204233 28303 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212: Generated new TSK 0
I20260811 18:44:25.205148 28303 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.185.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
I20260811 18:44:25.301805 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:36929
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:44:25.690212 28319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:44:25.690672 28319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:44:25.690877 28319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:44:25.704874 28319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:44:25.705060 28319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:44:25.705322 28319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:44:25.721119 28319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:36929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:44:25.723026 28319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:44:25.724388 28319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:44:25.740134 28329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:25.740192 28327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:25.740326 28331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:44:25.741036 28319 server_base.cc:1034] running on GCE node
I20260811 18:44:25.741952 28319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:44:25.743045 28319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:44:25.744287 28319 hybrid_clock.cc:648] HybridClock initialized: now 1786473865744220 us; error 51 us; skew 500 ppm
I20260811 18:44:25.744719 28319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:44:25 dist-test-slave-z8x9 krb5kdc[28129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473865, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:44:25.748477 28319 init.cc:377] Logged in from keytab as kudu/127.30.185.129@KRBTEST.COM (short username kudu)
I20260811 18:44:25.750329 28319 webserver.cc:460] Webserver started at http://127.30.185.129:45859/ using document root <none> and password file <none>
I20260811 18:44:25.751049 28319 fs_manager.cc:359] Metadata directory not provided
I20260811 18:44:25.751211 28319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:44:25.751547 28319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:44:25.754710 28319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "bd16c88da1cc42ab91697e45bd9e4bb3"
format_stamp: "Formatted at 2026-08-11 18:44:25 on dist-test-slave-z8x9"
server_key: "62ffffcdc8d47bea24c06b920a4a4db6"
server_key_iv: "f000db7f3ac4d793f4ccccd430906808"
server_key_version: "encryptionkey@0"
I20260811 18:44:25.755481 28319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "bd16c88da1cc42ab91697e45bd9e4bb3"
format_stamp: "Formatted at 2026-08-11 18:44:25 on dist-test-slave-z8x9"
server_key: "62ffffcdc8d47bea24c06b920a4a4db6"
server_key_iv: "f000db7f3ac4d793f4ccccd430906808"
server_key_version: "encryptionkey@0"
I20260811 18:44:25.761000 28319 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:44:25.764631 28342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:25.766067 28319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:44:25.766284 28319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "bd16c88da1cc42ab91697e45bd9e4bb3"
format_stamp: "Formatted at 2026-08-11 18:44:25 on dist-test-slave-z8x9"
server_key: "62ffffcdc8d47bea24c06b920a4a4db6"
server_key_iv: "f000db7f3ac4d793f4ccccd430906808"
server_key_version: "encryptionkey@0"
I20260811 18:44:25.766531 28319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:44:25.793802 28319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:44:25.798033 28319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:44:25.798374 28319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:44:25.799464 28319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:44:25.800895 28319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:44:25.801031 28319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:25.801146 28319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:44:25.801254 28319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 18:44:25.837222 28319 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:38077
I20260811 18:44:25.837376 28548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:38077 every 8 connection(s)
I20260811 18:44:25.838687 28319 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:44:25.842442 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 28319
I20260811 18:44:25.842680 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:44:25.843397 31462 external_mini_cluster.cc:1428] Setting key 48d5d5e7e2fe51c00eea41b82060679c
Aug 11 18:44:25 dist-test-slave-z8x9 krb5kdc[28129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473865, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:44:25.862766 28553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:25.841671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:57545 (local address 127.30.185.190:36929)
0811 18:44:25.842359 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:25.842367 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:25.843559 (+ 1192us) server_negotiation.cc:407] Connection header received
0811 18:44:25.845925 (+ 2366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:25.845930 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:25.846017 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:25.846157 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:25.847753 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:25.848968 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:25.850682 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:25.851414 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:25.851561 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:25.855262 (+ 3701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:25.855272 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:25.855280 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:25.855327 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:25.858985 (+ 3658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:25.860075 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:25.860079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:25.860081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:25.860282 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:25.860822 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:25.860826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:25.860828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:25.861115 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:25.861371 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:25.862330 (+ 959us) server_negotiation.cc:299] Negotiation successful
0811 18:44:25.862505 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":369,"threads_started":1}
I20260811 18:44:25.864374 28555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:25.842378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.129:57545)
0811 18:44:25.843300 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:25.843347 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:25.845571 (+ 2224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:25.846384 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:25.846392 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:25.846949 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:44:25.847549 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:25.847567 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:25.849126 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:25.849130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:25.850514 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:25.850533 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:25.851642 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:25.851647 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:25.851847 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:25.852692 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:44:25.852716 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:44:25.855013 (+ 2297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:44:25.859166 (+ 4153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:25.859179 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:25.859201 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:25.859935 (+ 734us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:25.860405 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:25.860412 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:25.860416 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:25.860693 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:25.861387 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:44:25.861400 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:44:25.861865 (+ 465us) client_negotiation.cc:769] Sending connection context
0811 18:44:25.862180 (+ 315us) client_negotiation.cc:240] Negotiation successful
0811 18:44:25.862799 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":425,"threads_started":1}
I20260811 18:44:25.866119 28551 heartbeater.cc:347] Connected to a master server at 127.30.185.190:36929
I20260811 18:44:25.866475 28551 heartbeater.cc:464] Registering TS with master...
I20260811 18:44:25.867249 28551 heartbeater.cc:511] Master 127.30.185.190:36929 requested a full tablet report, sending...
I20260811 18:44:25.869545 28212 ts_manager.cc:194] Registered new tserver with Master: bd16c88da1cc42ab91697e45bd9e4bb3 (127.30.185.129:38077)
I20260811 18:44:25.871352 28212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:57545
WARNING: no policy specified for kudu/127.30.185.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
I20260811 18:44:25.917412 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:36929
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:44:26.276963 28559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:44:26.277462 28559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:44:26.277650 28559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:44:26.285205 28559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:44:26.285387 28559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:44:26.285583 28559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:44:26.295518 28559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:36929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:44:26.296983 28559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:44:26.298449 28559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:44:26.312103 28568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:26.312208 28575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:26.312172 28569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:44:26.312862 28559 server_base.cc:1034] running on GCE node
I20260811 18:44:26.313557 28559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:44:26.314451 28559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:44:26.315665 28559 hybrid_clock.cc:648] HybridClock initialized: now 1786473866315624 us; error 31 us; skew 500 ppm
I20260811 18:44:26.315953 28559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:44:26 dist-test-slave-z8x9 krb5kdc[28129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473866, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:44:26.319326 28559 init.cc:377] Logged in from keytab as kudu/127.30.185.130@KRBTEST.COM (short username kudu)
I20260811 18:44:26.320704 28559 webserver.cc:460] Webserver started at http://127.30.185.130:34273/ using document root <none> and password file <none>
I20260811 18:44:26.321462 28559 fs_manager.cc:359] Metadata directory not provided
I20260811 18:44:26.321570 28559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:44:26.321837 28559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:44:26.324796 28559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "9872f13b5ff546778e9baff77e240ce1"
format_stamp: "Formatted at 2026-08-11 18:44:26 on dist-test-slave-z8x9"
server_key: "0d9d3310df96a5e5db55569df098236d"
server_key_iv: "e5a4a9d372f8b188f2e567b6ada93882"
server_key_version: "encryptionkey@0"
I20260811 18:44:26.325604 28559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "9872f13b5ff546778e9baff77e240ce1"
format_stamp: "Formatted at 2026-08-11 18:44:26 on dist-test-slave-z8x9"
server_key: "0d9d3310df96a5e5db55569df098236d"
server_key_iv: "e5a4a9d372f8b188f2e567b6ada93882"
server_key_version: "encryptionkey@0"
I20260811 18:44:26.330829 28559 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 18:44:26.334313 28587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:26.335625 28559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:44:26.335813 28559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "9872f13b5ff546778e9baff77e240ce1"
format_stamp: "Formatted at 2026-08-11 18:44:26 on dist-test-slave-z8x9"
server_key: "0d9d3310df96a5e5db55569df098236d"
server_key_iv: "e5a4a9d372f8b188f2e567b6ada93882"
server_key_version: "encryptionkey@0"
I20260811 18:44:26.336012 28559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:44:26.354940 28559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:44:26.359799 28559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:44:26.360118 28559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:44:26.361155 28559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:44:26.362666 28559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:44:26.362742 28559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:26.362892 28559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:44:26.362946 28559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:26.398437 28559 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:40149
I20260811 18:44:26.398479 28797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:40149 every 8 connection(s)
I20260811 18:44:26.399857 28559 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:44:26.410022 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 28559
I20260811 18:44:26.410358 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:44:26.410833 31462 external_mini_cluster.cc:1428] Setting key 27b7193af5bc8fcff17f7cb7dab20947
Aug 11 18:44:26 dist-test-slave-z8x9 krb5kdc[28129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473866, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:44:26.421820 28809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:26.402481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:53631 (local address 127.30.185.190:36929)
0811 18:44:26.402944 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:26.402949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:26.403769 (+ 820us) server_negotiation.cc:407] Connection header received
0811 18:44:26.405684 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:26.405690 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:26.405786 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:26.405914 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:26.407473 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:26.408201 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:26.409897 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:26.410397 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:26.410547 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:26.414472 (+ 3925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:26.414483 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:26.414487 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:26.414522 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:26.417858 (+ 3336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:26.418902 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:26.418909 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:26.418915 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:26.419077 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:26.419605 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:26.419612 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:26.419617 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:26.419975 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:26.420113 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:26.421079 (+ 966us) server_negotiation.cc:299] Negotiation successful
0811 18:44:26.421358 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":252,"threads_started":1}
I20260811 18:44:26.423133 28810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:26.402887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.130:53631)
0811 18:44:26.403602 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:26.403639 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:26.405401 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:26.406159 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:26.406167 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:26.406718 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:44:26.407285 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:26.407302 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:26.408361 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:26.408367 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:26.409735 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:26.409748 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:26.410668 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:26.410677 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:26.410922 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:26.411748 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:44:26.411788 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:44:26.414216 (+ 2428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:44:26.418019 (+ 3803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:26.418031 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:26.418054 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:26.418763 (+ 709us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:26.419203 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:26.419211 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:26.419217 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:26.419480 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:26.420118 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:44:26.420129 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:44:26.420632 (+ 503us) client_negotiation.cc:769] Sending connection context
0811 18:44:26.420967 (+ 335us) client_negotiation.cc:240] Negotiation successful
0811 18:44:26.421602 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":275,"threads_started":1}
I20260811 18:44:26.424690 28800 heartbeater.cc:347] Connected to a master server at 127.30.185.190:36929
I20260811 18:44:26.425019 28800 heartbeater.cc:464] Registering TS with master...
I20260811 18:44:26.425851 28800 heartbeater.cc:511] Master 127.30.185.190:36929 requested a full tablet report, sending...
I20260811 18:44:26.427486 28212 ts_manager.cc:194] Registered new tserver with Master: 9872f13b5ff546778e9baff77e240ce1 (127.30.185.130:40149)
I20260811 18:44:26.428283 28212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:53631
WARNING: no policy specified for kudu/127.30.185.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
I20260811 18:44:26.485291 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:36929
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:44:26.846582 28818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:44:26.846899 28818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:44:26.847070 28818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:44:26.855180 28818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:44:26.855345 28818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:44:26.855561 28818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:44:26.866133 28818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:36929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:44:26.867478 28818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:44:26.868793 28818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:44:26.874656 28551 heartbeater.cc:503] Master 127.30.185.190:36929 was elected leader, sending a full tablet report...
W20260811 18:44:26.883160 28828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:26.883350 28825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:26.883180 28826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:44:26.883935 28818 server_base.cc:1034] running on GCE node
I20260811 18:44:26.884774 28818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:44:26.885944 28818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:44:26.887179 28818 hybrid_clock.cc:648] HybridClock initialized: now 1786473866887135 us; error 61 us; skew 500 ppm
I20260811 18:44:26.887512 28818 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:44:26 dist-test-slave-z8x9 krb5kdc[28129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473866, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:44:26.890823 28818 init.cc:377] Logged in from keytab as kudu/127.30.185.131@KRBTEST.COM (short username kudu)
I20260811 18:44:26.892299 28818 webserver.cc:460] Webserver started at http://127.30.185.131:43589/ using document root <none> and password file <none>
I20260811 18:44:26.892994 28818 fs_manager.cc:359] Metadata directory not provided
I20260811 18:44:26.893141 28818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:44:26.893486 28818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:44:26.896421 28818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "72943d582ff9476e805b085051c10f2a"
format_stamp: "Formatted at 2026-08-11 18:44:26 on dist-test-slave-z8x9"
server_key: "4d3e2b6b07c32ea95f0043890caa9588"
server_key_iv: "375bec15ed57f32ad65a57963e26b476"
server_key_version: "encryptionkey@0"
I20260811 18:44:26.897275 28818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "72943d582ff9476e805b085051c10f2a"
format_stamp: "Formatted at 2026-08-11 18:44:26 on dist-test-slave-z8x9"
server_key: "4d3e2b6b07c32ea95f0043890caa9588"
server_key_iv: "375bec15ed57f32ad65a57963e26b476"
server_key_version: "encryptionkey@0"
I20260811 18:44:26.902618 28818 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 18:44:26.906177 28841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:26.907661 28818 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:44:26.907800 28818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "72943d582ff9476e805b085051c10f2a"
format_stamp: "Formatted at 2026-08-11 18:44:26 on dist-test-slave-z8x9"
server_key: "4d3e2b6b07c32ea95f0043890caa9588"
server_key_iv: "375bec15ed57f32ad65a57963e26b476"
server_key_version: "encryptionkey@0"
I20260811 18:44:26.907995 28818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:44:26.919461 28818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:44:26.924139 28818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:44:26.924422 28818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:44:26.925431 28818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:44:26.926765 28818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:44:26.926851 28818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:26.926936 28818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:44:26.926983 28818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:26.963021 28818 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:37527
I20260811 18:44:26.963085 29043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:37527 every 8 connection(s)
I20260811 18:44:26.964445 28818 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:44:26.968245 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 28818
I20260811 18:44:26.968478 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:44:26.969012 31462 external_mini_cluster.cc:1428] Setting key 671401412de90483752a69a32680bfa2
Aug 11 18:44:26 dist-test-slave-z8x9 krb5kdc[28129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473866, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:44:26.987898 29047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:26.967213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:41155 (local address 127.30.185.190:36929)
0811 18:44:26.967696 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:26.967701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:26.968920 (+ 1219us) server_negotiation.cc:407] Connection header received
0811 18:44:26.971463 (+ 2543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:26.971470 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:26.971589 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:26.971703 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:26.973673 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:26.974516 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:26.976277 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:26.976686 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:26.976814 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:26.980600 (+ 3786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:26.980605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:26.980609 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:26.980645 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:26.983754 (+ 3109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:26.984833 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:26.984837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:26.984840 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:26.985040 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:26.985671 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:26.985679 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:26.985681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:26.986066 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:26.986232 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:26.987241 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0811 18:44:26.987499 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":240,"threads_started":1}
I20260811 18:44:26.989130 29051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:26.967867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.131:41155)
0811 18:44:26.968709 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:26.968769 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:26.970990 (+ 2221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:26.972005 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:26.972014 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:26.972839 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:44:26.973500 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:26.973521 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:26.974666 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:26.974670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:26.976098 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:26.976117 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:26.976937 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:26.976943 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:26.977250 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:26.977980 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:44:26.978009 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:44:26.980354 (+ 2345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:44:26.983955 (+ 3601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:26.983970 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:26.983990 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:26.984669 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:26.985227 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:26.985234 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:26.985238 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:26.985521 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:26.986316 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:44:26.986325 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:44:26.986827 (+ 502us) client_negotiation.cc:769] Sending connection context
0811 18:44:26.987123 (+ 296us) client_negotiation.cc:240] Negotiation successful
0811 18:44:26.987690 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":352,"threads_started":1}
I20260811 18:44:26.990538 29044 heartbeater.cc:347] Connected to a master server at 127.30.185.190:36929
I20260811 18:44:26.990876 29044 heartbeater.cc:464] Registering TS with master...
I20260811 18:44:26.991597 29044 heartbeater.cc:511] Master 127.30.185.190:36929 requested a full tablet report, sending...
I20260811 18:44:26.993057 28212 ts_manager.cc:194] Registered new tserver with Master: 72943d582ff9476e805b085051c10f2a (127.30.185.131:37527)
I20260811 18:44:26.993948 28212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:41155
I20260811 18:44:26.998205 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:44:27.016724 29047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:27.003175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36674 (local address 127.30.185.190:36929)
0811 18:44:27.003454 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:27.003462 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:27.003774 (+ 312us) server_negotiation.cc:407] Connection header received
0811 18:44:27.003915 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:27.003919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:27.003989 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:27.004079 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:27.005668 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.006442 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.007878 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.008287 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.008385 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:27.010851 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:27.010856 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:27.010860 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:27.010896 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:27.013917 (+ 3021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:27.014623 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:27.014628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:27.014630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:27.014756 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:27.015138 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:27.015143 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:27.015146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:27.015595 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:27.015835 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:27.016131 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 18:44:27.016340 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 18:44:27.021405 28212 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36674:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:44:27.028501 28212 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:44:27.055066 29063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:27.041734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:40149 (local address 127.0.0.1:47390)
0811 18:44:27.042763 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:27.042824 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:27.043031 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:27.044645 (+ 1614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:27.044654 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:27.044696 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:44:27.045212 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:27.045230 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.047679 (+ 2449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.047687 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:27.050725 (+ 3038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:27.050746 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.053614 (+ 2868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.053624 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:27.053945 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:27.054019 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 18:44:27.054205 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 18:44:27.054478 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":514,"threads_started":1}
I20260811 18:44:27.056327 29068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:27.041833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47390 (local address 127.30.185.130:40149)
0811 18:44:27.043628 (+ 1795us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:27.043641 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:27.043676 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:44:27.043803 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:27.043814 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:27.044091 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:27.044317 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:44:27.045482 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.047470 (+ 1988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.050971 (+ 3501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.052921 (+ 1950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.053163 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:27.053439 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:27.055612 (+ 2173us) server_negotiation.cc:299] Negotiation successful
0811 18:44:27.055855 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1615,"thread_start_us":1507,"threads_started":1}
I20260811 18:44:27.062057 28681 tablet_service.cc:1467] Processing CreateTablet for tablet 05c57175facf4ad7a35cf8c96980dff8 (DEFAULT_TABLE table=test-table [id=e6471dc918e64dc2b3cfabd95192c938]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:44:27.063558 28681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05c57175facf4ad7a35cf8c96980dff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:44:27.065622 29070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:27.041852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:37527 (local address 127.0.0.1:56566)
0811 18:44:27.052802 (+ 10950us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:27.052831 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:27.053019 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:27.057101 (+ 4082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:27.057106 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:27.057150 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:44:27.057527 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:27.057540 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.060076 (+ 2536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.060082 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:27.062474 (+ 2392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:27.062489 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.064595 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.064601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:27.064796 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:27.064842 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 18:44:27.064926 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 18:44:27.065152 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10767,"mutex_wait_us":76,"thread_start_us":10571,"threads_started":1}
I20260811 18:44:27.065726 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:27.042499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56566 (local address 127.30.185.131:37527)
0811 18:44:27.055341 (+ 12842us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:27.055354 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:27.055402 (+ 48us) server_negotiation.cc:407] Connection header received
0811 18:44:27.055534 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:27.055547 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:27.055909 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:27.057869 (+ 1960us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:44:27.058137 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.059917 (+ 1780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.062658 (+ 2741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.064105 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.064327 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:27.064489 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:27.065156 (+ 667us) server_negotiation.cc:299] Negotiation successful
0811 18:44:27.065380 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12615,"thread_start_us":14317,"threads_started":1}
I20260811 18:44:27.066210 29075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:27.043397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48878 (local address 127.30.185.129:38077)
0811 18:44:27.057571 (+ 14174us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:27.057577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:27.057601 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:44:27.057684 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:27.057690 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:27.058103 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:27.058345 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:44:27.059037 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.060874 (+ 1837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.063234 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.064643 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.064844 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:27.065006 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:27.065730 (+ 724us) server_negotiation.cc:299] Negotiation successful
0811 18:44:27.065888 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14004,"thread_start_us":13858,"threads_started":1}
I20260811 18:44:27.066612 29073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:27.043134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:38077 (local address 127.0.0.1:48878)
0811 18:44:27.056577 (+ 13443us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:27.056601 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:27.056726 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:27.058363 (+ 1637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:27.058370 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:27.058402 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:44:27.058851 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:27.058865 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.061149 (+ 2284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.061154 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:27.063071 (+ 1917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:27.063082 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.064884 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.064894 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:27.065199 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:27.065230 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 18:44:27.065329 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 18:44:27.065446 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13325,"mutex_wait_us":72,"thread_start_us":13171,"threads_started":1}
I20260811 18:44:27.069726 28434 tablet_service.cc:1467] Processing CreateTablet for tablet 05c57175facf4ad7a35cf8c96980dff8 (DEFAULT_TABLE table=test-table [id=e6471dc918e64dc2b3cfabd95192c938]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:44:27.069811 28931 tablet_service.cc:1467] Processing CreateTablet for tablet 05c57175facf4ad7a35cf8c96980dff8 (DEFAULT_TABLE table=test-table [id=e6471dc918e64dc2b3cfabd95192c938]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:44:27.071434 28434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05c57175facf4ad7a35cf8c96980dff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:44:27.071537 28931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05c57175facf4ad7a35cf8c96980dff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:44:27.081967 29079 tablet_bootstrap.cc:492] T 05c57175facf4ad7a35cf8c96980dff8 P 9872f13b5ff546778e9baff77e240ce1: Bootstrap starting.
I20260811 18:44:27.084910 29083 tablet_bootstrap.cc:492] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3: Bootstrap starting.
I20260811 18:44:27.086812 29084 tablet_bootstrap.cc:492] T 05c57175facf4ad7a35cf8c96980dff8 P 72943d582ff9476e805b085051c10f2a: Bootstrap starting.
I20260811 18:44:27.087750 29079 tablet_bootstrap.cc:654] T 05c57175facf4ad7a35cf8c96980dff8 P 9872f13b5ff546778e9baff77e240ce1: Neither blocks nor log segments found. Creating new log.
I20260811 18:44:27.088435 29083 tablet_bootstrap.cc:654] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3: Neither blocks nor log segments found. Creating new log.
I20260811 18:44:27.089128 29079 log.cc:824] T 05c57175facf4ad7a35cf8c96980dff8 P 9872f13b5ff546778e9baff77e240ce1: Log is configured to *not* fsync() on all Append() calls
I20260811 18:44:27.089526 29083 log.cc:824] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3: Log is configured to *not* fsync() on all Append() calls
I20260811 18:44:27.090317 29084 tablet_bootstrap.cc:654] T 05c57175facf4ad7a35cf8c96980dff8 P 72943d582ff9476e805b085051c10f2a: Neither blocks nor log segments found. Creating new log.
I20260811 18:44:27.091485 29084 log.cc:824] T 05c57175facf4ad7a35cf8c96980dff8 P 72943d582ff9476e805b085051c10f2a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:44:27.092768 29083 tablet_bootstrap.cc:492] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3: No bootstrap required, opened a new log
I20260811 18:44:27.092764 29079 tablet_bootstrap.cc:492] T 05c57175facf4ad7a35cf8c96980dff8 P 9872f13b5ff546778e9baff77e240ce1: No bootstrap required, opened a new log
I20260811 18:44:27.093053 29083 ts_tablet_manager.cc:1397] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 18:44:27.093053 29079 ts_tablet_manager.cc:1397] T 05c57175facf4ad7a35cf8c96980dff8 P 9872f13b5ff546778e9baff77e240ce1: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 18:44:27.095953 29084 tablet_bootstrap.cc:492] T 05c57175facf4ad7a35cf8c96980dff8 P 72943d582ff9476e805b085051c10f2a: No bootstrap required, opened a new log
I20260811 18:44:27.096232 29084 ts_tablet_manager.cc:1397] T 05c57175facf4ad7a35cf8c96980dff8 P 72943d582ff9476e805b085051c10f2a: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 18:44:27.104847 29083 raft_consensus.cc:348] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9872f13b5ff546778e9baff77e240ce1" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40149 } } peers { permanent_uuid: "72943d582ff9476e805b085051c10f2a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37527 } } peers { permanent_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38077 } }
I20260811 18:44:27.104905 29079 raft_consensus.cc:348] T 05c57175facf4ad7a35cf8c96980dff8 P 9872f13b5ff546778e9baff77e240ce1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9872f13b5ff546778e9baff77e240ce1" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40149 } } peers { permanent_uuid: "72943d582ff9476e805b085051c10f2a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37527 } } peers { permanent_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38077 } }
I20260811 18:44:27.105425 29079 raft_consensus.cc:374] T 05c57175facf4ad7a35cf8c96980dff8 P 9872f13b5ff546778e9baff77e240ce1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:44:27.105423 29083 raft_consensus.cc:374] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:44:27.105531 29079 raft_consensus.cc:729] T 05c57175facf4ad7a35cf8c96980dff8 P 9872f13b5ff546778e9baff77e240ce1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9872f13b5ff546778e9baff77e240ce1, State: Initialized, Role: FOLLOWER
I20260811 18:44:27.105532 29083 raft_consensus.cc:729] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd16c88da1cc42ab91697e45bd9e4bb3, State: Initialized, Role: FOLLOWER
I20260811 18:44:27.106235 29083 consensus_queue.cc:260] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9872f13b5ff546778e9baff77e240ce1" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40149 } } peers { permanent_uuid: "72943d582ff9476e805b085051c10f2a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37527 } } peers { permanent_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38077 } }
I20260811 18:44:27.106244 29079 consensus_queue.cc:260] T 05c57175facf4ad7a35cf8c96980dff8 P 9872f13b5ff546778e9baff77e240ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9872f13b5ff546778e9baff77e240ce1" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40149 } } peers { permanent_uuid: "72943d582ff9476e805b085051c10f2a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37527 } } peers { permanent_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38077 } }
I20260811 18:44:27.107471 29084 raft_consensus.cc:348] T 05c57175facf4ad7a35cf8c96980dff8 P 72943d582ff9476e805b085051c10f2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9872f13b5ff546778e9baff77e240ce1" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40149 } } peers { permanent_uuid: "72943d582ff9476e805b085051c10f2a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37527 } } peers { permanent_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38077 } }
I20260811 18:44:27.107921 29084 raft_consensus.cc:374] T 05c57175facf4ad7a35cf8c96980dff8 P 72943d582ff9476e805b085051c10f2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:44:27.108021 29084 raft_consensus.cc:729] T 05c57175facf4ad7a35cf8c96980dff8 P 72943d582ff9476e805b085051c10f2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72943d582ff9476e805b085051c10f2a, State: Initialized, Role: FOLLOWER
I20260811 18:44:27.108742 29084 consensus_queue.cc:260] T 05c57175facf4ad7a35cf8c96980dff8 P 72943d582ff9476e805b085051c10f2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9872f13b5ff546778e9baff77e240ce1" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40149 } } peers { permanent_uuid: "72943d582ff9476e805b085051c10f2a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37527 } } peers { permanent_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38077 } }
I20260811 18:44:27.115150 29083 ts_tablet_manager.cc:1428] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3: Time spent starting tablet: real 0.022s user 0.017s sys 0.005s
I20260811 18:44:27.116900 29084 ts_tablet_manager.cc:1428] T 05c57175facf4ad7a35cf8c96980dff8 P 72943d582ff9476e805b085051c10f2a: Time spent starting tablet: real 0.021s user 0.016s sys 0.004s
I20260811 18:44:27.117323 29044 heartbeater.cc:503] Master 127.30.185.190:36929 was elected leader, sending a full tablet report...
I20260811 18:44:27.120405 28800 heartbeater.cc:503] Master 127.30.185.190:36929 was elected leader, sending a full tablet report...
I20260811 18:44:27.122117 29079 ts_tablet_manager.cc:1428] T 05c57175facf4ad7a35cf8c96980dff8 P 9872f13b5ff546778e9baff77e240ce1: Time spent starting tablet: real 0.029s user 0.014s sys 0.013s
W20260811 18:44:27.152040 28807 tablet.cc:2369] T 05c57175facf4ad7a35cf8c96980dff8 P 9872f13b5ff546778e9baff77e240ce1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:44:27.216252 29045 tablet.cc:2369] T 05c57175facf4ad7a35cf8c96980dff8 P 72943d582ff9476e805b085051c10f2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:44:27.341511 28552 tablet.cc:2369] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:44:27.408239 29089 raft_consensus.cc:482] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:44:27.408596 29089 raft_consensus.cc:504] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9872f13b5ff546778e9baff77e240ce1" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40149 } } peers { permanent_uuid: "72943d582ff9476e805b085051c10f2a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37527 } } peers { permanent_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38077 } }
I20260811 18:44:27.410542 29089 leader_election.cc:288] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9872f13b5ff546778e9baff77e240ce1 (127.30.185.130:40149), 72943d582ff9476e805b085051c10f2a (127.30.185.131:37527)
I20260811 18:44:27.418105 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:27.411355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:55331 (local address 127.30.185.131:37527)
0811 18:44:27.411569 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:27.411576 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:27.411689 (+ 113us) server_negotiation.cc:407] Connection header received
0811 18:44:27.412198 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:27.412205 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:27.412287 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:27.412404 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:44:27.413031 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.414045 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.416112 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.417005 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.417273 (+ 268us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:27.417359 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:27.417673 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 18:44:27.417824 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 18:44:27.418284 29093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:27.410860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:37527 (local address 127.30.185.129:55331)
0811 18:44:27.411569 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:27.411593 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:27.412013 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:27.412428 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:27.412433 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:27.412464 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:44:27.412867 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:27.412878 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.414198 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.414205 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:27.415922 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:27.415938 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.417300 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.417307 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:27.417489 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:27.417516 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 18:44:27.417644 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 18:44:27.417792 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":793,"threads_started":1}
I20260811 18:44:27.419376 28957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05c57175facf4ad7a35cf8c96980dff8" candidate_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72943d582ff9476e805b085051c10f2a" is_pre_election: true
I20260811 18:44:27.419950 28957 raft_consensus.cc:2454] T 05c57175facf4ad7a35cf8c96980dff8 P 72943d582ff9476e805b085051c10f2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd16c88da1cc42ab91697e45bd9e4bb3 in term 0.
I20260811 18:44:27.420657 28347 leader_election.cc:302] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72943d582ff9476e805b085051c10f2a, bd16c88da1cc42ab91697e45bd9e4bb3; no voters:
I20260811 18:44:27.421092 29089 raft_consensus.cc:2781] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:44:27.421296 29089 raft_consensus.cc:482] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:44:27.421434 29089 raft_consensus.cc:3037] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:44:27.422170 29068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:27.411709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:50127 (local address 127.30.185.130:40149)
0811 18:44:27.411923 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:27.411928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:27.415932 (+ 4004us) server_negotiation.cc:407] Connection header received
0811 18:44:27.416159 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:27.416164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:27.416226 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:27.416340 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:44:27.416873 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.417684 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.420028 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.421135 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.421457 (+ 322us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:27.421563 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:27.421872 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 18:44:27.421977 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:44:27.422395 29095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:27.410982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:40149 (local address 127.30.185.129:50127)
0811 18:44:27.415803 (+ 4821us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:27.415821 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:27.415999 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:27.416355 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:27.416359 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:27.416384 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:44:27.416715 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:27.416724 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.417828 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.417835 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:27.419855 (+ 2020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:27.419873 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.421480 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.421487 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:27.421676 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:27.421717 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 18:44:27.421840 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 18:44:27.421981 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4678,"thread_start_us":4495,"threads_started":1}
I20260811 18:44:27.424043 29089 raft_consensus.cc:504] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9872f13b5ff546778e9baff77e240ce1" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40149 } } peers { permanent_uuid: "72943d582ff9476e805b085051c10f2a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37527 } } peers { permanent_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38077 } }
I20260811 18:44:27.424664 29089 leader_election.cc:288] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3 [CANDIDATE]: Term 1 election: Requested vote from peers 9872f13b5ff546778e9baff77e240ce1 (127.30.185.130:40149), 72943d582ff9476e805b085051c10f2a (127.30.185.131:37527)
I20260811 18:44:27.425282 28957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05c57175facf4ad7a35cf8c96980dff8" candidate_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72943d582ff9476e805b085051c10f2a"
I20260811 18:44:27.425520 28957 raft_consensus.cc:3037] T 05c57175facf4ad7a35cf8c96980dff8 P 72943d582ff9476e805b085051c10f2a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:44:27.425850 28717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05c57175facf4ad7a35cf8c96980dff8" candidate_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9872f13b5ff546778e9baff77e240ce1"
I20260811 18:44:27.425899 28721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05c57175facf4ad7a35cf8c96980dff8" candidate_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9872f13b5ff546778e9baff77e240ce1" is_pre_election: true
I20260811 18:44:27.426435 28717 raft_consensus.cc:3037] T 05c57175facf4ad7a35cf8c96980dff8 P 9872f13b5ff546778e9baff77e240ce1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:44:27.428845 28957 raft_consensus.cc:2454] T 05c57175facf4ad7a35cf8c96980dff8 P 72943d582ff9476e805b085051c10f2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd16c88da1cc42ab91697e45bd9e4bb3 in term 1.
I20260811 18:44:27.429312 28347 leader_election.cc:302] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72943d582ff9476e805b085051c10f2a, bd16c88da1cc42ab91697e45bd9e4bb3; no voters:
I20260811 18:44:27.429663 29089 raft_consensus.cc:2781] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:44:27.429982 28717 raft_consensus.cc:2454] T 05c57175facf4ad7a35cf8c96980dff8 P 9872f13b5ff546778e9baff77e240ce1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd16c88da1cc42ab91697e45bd9e4bb3 in term 1.
I20260811 18:44:27.430649 29089 raft_consensus.cc:686] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3 [term 1 LEADER]: Becoming Leader. State: Replica: bd16c88da1cc42ab91697e45bd9e4bb3, State: Running, Role: LEADER
I20260811 18:44:27.431143 29089 consensus_queue.cc:237] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9872f13b5ff546778e9baff77e240ce1" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40149 } } peers { permanent_uuid: "72943d582ff9476e805b085051c10f2a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37527 } } peers { permanent_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38077 } }
I20260811 18:44:27.435879 28207 catalog_manager.cc:5495] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd16c88da1cc42ab91697e45bd9e4bb3 (127.30.185.129). New cstate: current_term: 1 leader_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9872f13b5ff546778e9baff77e240ce1" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72943d582ff9476e805b085051c10f2a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38077 } health_report { overall_health: HEALTHY } } }
I20260811 18:44:27.532420 29075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:27.526388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48880 (local address 127.30.185.129:38077)
0811 18:44:27.526632 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:27.526638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:27.526802 (+ 164us) server_negotiation.cc:407] Connection header received
0811 18:44:27.527068 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:27.527072 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:27.527165 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:27.527342 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:44:27.527783 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.528524 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.530115 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.530547 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.530703 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:27.531280 (+ 577us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:44:27.531808 (+ 528us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:44:27.531981 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:27.532039 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 18:44:27.532119 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 18:44:27.541517 28717 raft_consensus.cc:1261] T 05c57175facf4ad7a35cf8c96980dff8 P 9872f13b5ff546778e9baff77e240ce1 [term 1 FOLLOWER]: Refusing update from remote peer bd16c88da1cc42ab91697e45bd9e4bb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:44:27.541536 28957 raft_consensus.cc:1261] T 05c57175facf4ad7a35cf8c96980dff8 P 72943d582ff9476e805b085051c10f2a [term 1 FOLLOWER]: Refusing update from remote peer bd16c88da1cc42ab91697e45bd9e4bb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:44:27.542443 29096 consensus_queue.cc:1035] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9872f13b5ff546778e9baff77e240ce1" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:44:27.542804 29089 consensus_queue.cc:1035] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72943d582ff9476e805b085051c10f2a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:44:27.571339 28206 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:44:27.571659 28206 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:44:27.575295 28206 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212: Sending DeleteTablet for 3 replicas of tablet 05c57175facf4ad7a35cf8c96980dff8
I20260811 18:44:27.576534 28681 tablet_service.cc:1514] Processing DeleteTablet for tablet 05c57175facf4ad7a35cf8c96980dff8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:44:27 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:47390
I20260811 18:44:27.576589 28931 tablet_service.cc:1514] Processing DeleteTablet for tablet 05c57175facf4ad7a35cf8c96980dff8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:44:27 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:56566
I20260811 18:44:27.577024 28434 tablet_service.cc:1514] Processing DeleteTablet for tablet 05c57175facf4ad7a35cf8c96980dff8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:44:27 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:48878
I20260811 18:44:27.577930 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 28319
I20260811 18:44:27.589000 29117 tablet_replica.cc:330] T 05c57175facf4ad7a35cf8c96980dff8 P 72943d582ff9476e805b085051c10f2a: stopping tablet replica
I20260811 18:44:27.589800 29118 tablet_replica.cc:330] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3: stopping tablet replica
I20260811 18:44:27.590427 29118 raft_consensus.cc:2229] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:44:27.591221 29118 raft_consensus.cc:2258] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:44:27.592271 29117 raft_consensus.cc:2229] T 05c57175facf4ad7a35cf8c96980dff8 P 72943d582ff9476e805b085051c10f2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:44:27.592908 29117 raft_consensus.cc:2258] T 05c57175facf4ad7a35cf8c96980dff8 P 72943d582ff9476e805b085051c10f2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:44:27.595366 29117 ts_tablet_manager.cc:1905] T 05c57175facf4ad7a35cf8c96980dff8 P 72943d582ff9476e805b085051c10f2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:44:27.595494 29118 ts_tablet_manager.cc:1905] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:44:27.597653 29116 tablet_replica.cc:330] T 05c57175facf4ad7a35cf8c96980dff8 P 9872f13b5ff546778e9baff77e240ce1: stopping tablet replica
I20260811 18:44:27.598114 29116 raft_consensus.cc:2229] T 05c57175facf4ad7a35cf8c96980dff8 P 9872f13b5ff546778e9baff77e240ce1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:44:27.598663 29116 raft_consensus.cc:2258] T 05c57175facf4ad7a35cf8c96980dff8 P 9872f13b5ff546778e9baff77e240ce1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:44:27.600827 29116 ts_tablet_manager.cc:1905] T 05c57175facf4ad7a35cf8c96980dff8 P 9872f13b5ff546778e9baff77e240ce1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:44:27.605269 29118 ts_tablet_manager.cc:1918] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:44:27.605532 29118 log.cc:1196] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/wal/wals/05c57175facf4ad7a35cf8c96980dff8
I20260811 18:44:27.606226 29118 ts_tablet_manager.cc:1939] T 05c57175facf4ad7a35cf8c96980dff8 P bd16c88da1cc42ab91697e45bd9e4bb3: Deleting consensus metadata
I20260811 18:44:27.608105 29116 ts_tablet_manager.cc:1918] T 05c57175facf4ad7a35cf8c96980dff8 P 9872f13b5ff546778e9baff77e240ce1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:44:27.608291 29116 log.cc:1196] T 05c57175facf4ad7a35cf8c96980dff8 P 9872f13b5ff546778e9baff77e240ce1: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/05c57175facf4ad7a35cf8c96980dff8
I20260811 18:44:27.608599 28184 catalog_manager.cc:4841] TS bd16c88da1cc42ab91697e45bd9e4bb3 (127.30.185.129:38077): tablet 05c57175facf4ad7a35cf8c96980dff8 (table test-table [id=e6471dc918e64dc2b3cfabd95192c938]) successfully deleted
I20260811 18:44:27.608850 29116 ts_tablet_manager.cc:1939] T 05c57175facf4ad7a35cf8c96980dff8 P 9872f13b5ff546778e9baff77e240ce1: Deleting consensus metadata
I20260811 18:44:27.609107 29117 ts_tablet_manager.cc:1918] T 05c57175facf4ad7a35cf8c96980dff8 P 72943d582ff9476e805b085051c10f2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:44:27.609367 29117 log.cc:1196] T 05c57175facf4ad7a35cf8c96980dff8 P 72943d582ff9476e805b085051c10f2a: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/05c57175facf4ad7a35cf8c96980dff8
I20260811 18:44:27.610008 29117 ts_tablet_manager.cc:1939] T 05c57175facf4ad7a35cf8c96980dff8 P 72943d582ff9476e805b085051c10f2a: Deleting consensus metadata
Aug 11 18:44:27 dist-test-slave-z8x9 krb5kdc[28129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.129@KRBTEST.COM
I20260811 18:44:27.611042 28184 catalog_manager.cc:4841] TS 9872f13b5ff546778e9baff77e240ce1 (127.30.185.130:40149): tablet 05c57175facf4ad7a35cf8c96980dff8 (table test-table [id=e6471dc918e64dc2b3cfabd95192c938]) successfully deleted
I20260811 18:44:27.611809 28185 catalog_manager.cc:4841] TS 72943d582ff9476e805b085051c10f2a (127.30.185.131:37527): tablet 05c57175facf4ad7a35cf8c96980dff8 (table test-table [id=e6471dc918e64dc2b3cfabd95192c938]) successfully deleted
I20260811 18:44:27.619614 29075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:27.579111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48894 (local address 127.30.185.129:38077)
0811 18:44:27.579325 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:27.579332 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:27.587183 (+ 7851us) server_negotiation.cc:407] Connection header received
0811 18:44:27.587436 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:27.587441 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:27.587537 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:27.587657 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:27.591350 (+ 3693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.592555 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.598980 (+ 6425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.599764 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.599895 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:27.612587 (+ 12692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:27.612601 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:27.612616 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:27.612659 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:27.616118 (+ 3459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:27.617112 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:27.617120 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:27.617122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:27.617390 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:27.617865 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:27.617868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:27.617870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:27.618212 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:27.618484 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:27.618923 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 18:44:27.619205 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 18:44:27.620486 28542 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:44:27.723965 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 28319
I20260811 18:44:27.754161 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 28559
Aug 11 18:44:27 dist-test-slave-z8x9 krb5kdc[28129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.130@KRBTEST.COM
I20260811 18:44:27.771544 29068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:27.755035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47404 (local address 127.30.185.130:40149)
0811 18:44:27.755223 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:27.755227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:27.755251 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:44:27.755429 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:27.755433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:27.755571 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:27.755676 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:27.757314 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.758030 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.759469 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.759875 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.760016 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:27.765304 (+ 5288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:27.765321 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:27.765337 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:27.765375 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:27.768282 (+ 2907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:27.769053 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:27.769062 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:27.769064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:27.769308 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:27.769822 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:27.769827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:27.769830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:27.770328 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:27.770599 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:27.771009 (+ 410us) server_negotiation.cc:299] Negotiation successful
0811 18:44:27.771243 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 18:44:27.772282 28790 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:44:27.867002 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 28559
I20260811 18:44:27.891667 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 28818
Aug 11 18:44:27 dist-test-slave-z8x9 krb5kdc[28129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.131@KRBTEST.COM
I20260811 18:44:27.908123 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:27.892522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56576 (local address 127.30.185.131:37527)
0811 18:44:27.892696 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:27.892701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:27.892720 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:44:27.892918 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:27.892922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:27.892989 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:27.893087 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:27.894801 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.895571 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.897119 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:27.897614 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:27.897715 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:27.902588 (+ 4873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:27.902604 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:27.902621 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:27.902656 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:27.905504 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:27.906167 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:27.906173 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:27.906176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:27.906311 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:27.906651 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:27.906654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:27.906656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:27.907008 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:27.907239 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:27.907559 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 18:44:27.907724 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 18:44:27.908855 29033 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:44:28.002287 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 28818
I20260811 18:44:28.026526 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 28153
I20260811 18:44:28.027789 28270 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:44:28.141553 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 28153
I20260811 18:44:48.180756 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:36929
--webserver_interface=127.30.185.190
--webserver_port=42741
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:36929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:44:48.533986 29130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:44:48.534292 29130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:44:48.534502 29130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:44:48.542687 29130 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 18:44:48.542800 29130 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 18:44:48.542883 29130 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 18:44:48.542986 29130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:44:48.543088 29130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:44:48.543208 29130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:44:48.543292 29130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:44:48.543372 29130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:44:48.553682 29130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:36929
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:36929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=42741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:44:48.555029 29130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:44:48.556392 29130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:44:48.582252 29144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:48.582300 29141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:48.582275 29142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:44:48.582863 29130 server_base.cc:1034] running on GCE node
I20260811 18:44:48.583899 29130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:44:48.585718 29130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:44:48.587019 29130 hybrid_clock.cc:648] HybridClock initialized: now 1786473888586944 us; error 64 us; skew 500 ppm
I20260811 18:44:48.587558 29130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:44:48 dist-test-slave-z8x9 krb5kdc[28129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473888, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:44:48.591856 29130 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:44:48.593714 29130 webserver.cc:460] Webserver started at http://127.30.185.190:42741/ using document root <none> and password file <none>
I20260811 18:44:48.594602 29130 fs_manager.cc:359] Metadata directory not provided
I20260811 18:44:48.594761 29130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:44:48.601845 29130 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 18:44:48.605124 29158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:48.606842 29130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:44:48.607056 29130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "5cd1365a8a1047d1bec51d03529ad212"
format_stamp: "Formatted at 2026-08-11 18:44:25 on dist-test-slave-z8x9"
server_key: "4a5725a31e5b88d8e5f411e7dc921749"
server_key_iv: "faa9a9d0311eef54f135095635648f82"
server_key_version: "encryptionkey@0"
I20260811 18:44:48.608224 29130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:44:48.629930 29130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:44:48.634692 29130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:44:48.634977 29130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:44:48.654035 29130 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:36929
I20260811 18:44:48.654062 29243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:36929 every 8 connection(s)
I20260811 18:44:48.655679 29130 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:44:48.657487 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 29130
I20260811 18:44:48.658623 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:38077
--local_ip_for_outbound_sockets=127.30.185.129
--tserver_master_addrs=127.30.185.190:36929
--webserver_port=45859
--webserver_interface=127.30.185.129
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:44:48.666635 29244 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:44:48.672327 29244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212: Bootstrap starting.
I20260811 18:44:48.695861 29244 log.cc:824] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212: Log is configured to *not* fsync() on all Append() calls
I20260811 18:44:48.709152 29244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:44:48.709860 29244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212: Bootstrap complete.
I20260811 18:44:48.719825 29244 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd1365a8a1047d1bec51d03529ad212" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 36929 } }
I20260811 18:44:48.720745 29244 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd1365a8a1047d1bec51d03529ad212, State: Initialized, Role: FOLLOWER
I20260811 18:44:48.721402 29244 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd1365a8a1047d1bec51d03529ad212" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 36929 } }
I20260811 18:44:48.721685 29244 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:44:48.721832 29244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:44:48.722024 29244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:44:48.726074 29244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd1365a8a1047d1bec51d03529ad212" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 36929 } }
I20260811 18:44:48.726640 29244 leader_election.cc:302] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cd1365a8a1047d1bec51d03529ad212; no voters:
I20260811 18:44:48.727324 29244 leader_election.cc:288] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 18:44:48.727476 29249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:44:48.728926 29249 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 2 LEADER]: Becoming Leader. State: Replica: 5cd1365a8a1047d1bec51d03529ad212, State: Running, Role: LEADER
I20260811 18:44:48.729626 29249 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd1365a8a1047d1bec51d03529ad212" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 36929 } }
I20260811 18:44:48.729869 29244 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:44:48.732280 29252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cd1365a8a1047d1bec51d03529ad212. Latest consensus state: current_term: 2 leader_uuid: "5cd1365a8a1047d1bec51d03529ad212" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd1365a8a1047d1bec51d03529ad212" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 36929 } } }
I20260811 18:44:48.732676 29252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [sys.catalog]: This master's current role is: LEADER
I20260811 18:44:48.732452 29251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5cd1365a8a1047d1bec51d03529ad212" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd1365a8a1047d1bec51d03529ad212" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 36929 } } }
I20260811 18:44:48.732870 29251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [sys.catalog]: This master's current role is: LEADER
I20260811 18:44:48.743297 29262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:44:48.751937 29262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:44:48.753463 29262 catalog_manager.cc:1259] Loaded cluster ID: e59b1719777b4493b26dbd931f8edc00
I20260811 18:44:48.753660 29262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:44:48.757886 29262 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:44:48.759542 29262 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212: Loaded TSK: 0
I20260811 18:44:48.799695 29262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212: Generated new TSK 1
I20260811 18:44:48.800102 29262 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 18:44:49.057718 29246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:44:49.058017 29246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:44:49.058179 29246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:44:49.065578 29246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:44:49.065694 29246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:44:49.065862 29246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:44:49.080263 29246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:38077
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=45859
--tserver_master_addrs=127.30.185.190:36929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:44:49.083616 29246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:44:49.085381 29246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:44:49.101814 29286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:49.101850 29287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:49.106452 29292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:44:49.107131 29246 server_base.cc:1034] running on GCE node
I20260811 18:44:49.107939 29246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:44:49.109225 29246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:44:49.110487 29246 hybrid_clock.cc:648] HybridClock initialized: now 1786473889110418 us; error 45 us; skew 500 ppm
I20260811 18:44:49.110882 29246 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:44:49 dist-test-slave-z8x9 krb5kdc[28129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473889, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:44:49.114866 29246 init.cc:377] Logged in from keytab as kudu/127.30.185.129@KRBTEST.COM (short username kudu)
I20260811 18:44:49.116726 29246 webserver.cc:460] Webserver started at http://127.30.185.129:45859/ using document root <none> and password file <none>
I20260811 18:44:49.117597 29246 fs_manager.cc:359] Metadata directory not provided
I20260811 18:44:49.117781 29246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:44:49.123980 29246 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:44:49.127043 29300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:49.128497 29246 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:44:49.128688 29246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "bd16c88da1cc42ab91697e45bd9e4bb3"
format_stamp: "Formatted at 2026-08-11 18:44:25 on dist-test-slave-z8x9"
server_key: "62ffffcdc8d47bea24c06b920a4a4db6"
server_key_iv: "f000db7f3ac4d793f4ccccd430906808"
server_key_version: "encryptionkey@0"
I20260811 18:44:49.129776 29246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:44:49.143994 29246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:44:49.148749 29246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:44:49.149050 29246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:44:49.150102 29246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:44:49.151500 29246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:44:49.151638 29246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:49.151768 29246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:44:49.151865 29246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:49.186378 29246 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:38077
I20260811 18:44:49.186437 29494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:38077 every 8 connection(s)
I20260811 18:44:49.188031 29246 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:44:49.197786 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 29246
I20260811 18:44:49.198534 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:40149
--local_ip_for_outbound_sockets=127.30.185.130
--tserver_master_addrs=127.30.185.190:36929
--webserver_port=34273
--webserver_interface=127.30.185.130
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:44:49 dist-test-slave-z8x9 krb5kdc[28129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473889, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:44:49.212591 29499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:49.191151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:49905 (local address 127.30.185.190:36929)
0811 18:44:49.191742 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:49.191754 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:49.192581 (+ 827us) server_negotiation.cc:407] Connection header received
0811 18:44:49.194864 (+ 2283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:49.194876 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:49.195193 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:49.195507 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:49.197020 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:49.198577 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:49.200335 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:49.201130 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:49.201384 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:49.204957 (+ 3573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:49.204966 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:49.204981 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:49.205023 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:49.208075 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:49.208849 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:49.208857 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:49.208862 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:49.209025 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:49.209496 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:49.209504 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:49.209507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:49.209981 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:49.210196 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:49.210997 (+ 801us) server_negotiation.cc:299] Negotiation successful
0811 18:44:49.211444 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":258,"threads_started":1}
I20260811 18:44:49.212759 29501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:49.191456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.129:49905)
0811 18:44:49.192382 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:49.192430 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:49.194314 (+ 1884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:49.195686 (+ 1372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:49.195694 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:49.196236 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:44:49.196815 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:49.196833 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:49.198747 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:49.198754 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:49.200175 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:49.200192 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:49.201401 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:49.201409 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:49.201694 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:49.202433 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:44:49.202455 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:44:49.204665 (+ 2210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:44:49.208228 (+ 3563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:49.208239 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:49.208254 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:49.208681 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:49.209152 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:49.209157 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:49.209161 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:49.209377 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:49.210131 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:44:49.210139 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:44:49.210473 (+ 334us) client_negotiation.cc:769] Sending connection context
0811 18:44:49.210779 (+ 306us) client_negotiation.cc:240] Negotiation successful
0811 18:44:49.211380 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":396,"threads_started":1}
I20260811 18:44:49.215873 29495 heartbeater.cc:347] Connected to a master server at 127.30.185.190:36929
I20260811 18:44:49.216228 29495 heartbeater.cc:464] Registering TS with master...
I20260811 18:44:49.217121 29495 heartbeater.cc:511] Master 127.30.185.190:36929 requested a full tablet report, sending...
I20260811 18:44:49.219525 29180 ts_manager.cc:194] Registered new tserver with Master: bd16c88da1cc42ab91697e45bd9e4bb3 (127.30.185.129:38077)
I20260811 18:44:49.221050 29180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:49905
W20260811 18:44:49.543990 29502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:44:49.544287 29502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:44:49.544466 29502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:44:49.552454 29502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:44:49.552572 29502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:44:49.552786 29502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:44:49.562665 29502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:40149
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=34273
--tserver_master_addrs=127.30.185.190:36929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:44:49.564015 29502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:44:49.565495 29502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:44:49.579020 29509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:49.578992 29507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:49.579468 29512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:44:49.579779 29502 server_base.cc:1034] running on GCE node
I20260811 18:44:49.580430 29502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:44:49.581416 29502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:44:49.582650 29502 hybrid_clock.cc:648] HybridClock initialized: now 1786473889582590 us; error 52 us; skew 500 ppm
I20260811 18:44:49.582970 29502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:44:49 dist-test-slave-z8x9 krb5kdc[28129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473889, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:44:49.586300 29502 init.cc:377] Logged in from keytab as kudu/127.30.185.130@KRBTEST.COM (short username kudu)
I20260811 18:44:49.587878 29502 webserver.cc:460] Webserver started at http://127.30.185.130:34273/ using document root <none> and password file <none>
I20260811 18:44:49.588577 29502 fs_manager.cc:359] Metadata directory not provided
I20260811 18:44:49.588752 29502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:44:49.594259 29502 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 18:44:49.597036 29520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:49.598431 29502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:44:49.598624 29502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "9872f13b5ff546778e9baff77e240ce1"
format_stamp: "Formatted at 2026-08-11 18:44:26 on dist-test-slave-z8x9"
server_key: "0d9d3310df96a5e5db55569df098236d"
server_key_iv: "e5a4a9d372f8b188f2e567b6ada93882"
server_key_version: "encryptionkey@0"
I20260811 18:44:49.599617 29502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:44:49.623862 29502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:44:49.628365 29502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:44:49.628659 29502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:44:49.629709 29502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:44:49.631114 29502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:44:49.631245 29502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:49.631382 29502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:44:49.631476 29502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:49.671557 29502 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:40149
I20260811 18:44:49.671605 29720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:40149 every 8 connection(s)
I20260811 18:44:49.673010 29502 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:44:49.673794 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 29502
I20260811 18:44:49.674636 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:37527
--local_ip_for_outbound_sockets=127.30.185.131
--tserver_master_addrs=127.30.185.190:36929
--webserver_port=43589
--webserver_interface=127.30.185.131
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:44:49 dist-test-slave-z8x9 krb5kdc[28129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473889, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:44:49.693853 29499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:49.675988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:54279 (local address 127.30.185.190:36929)
0811 18:44:49.676189 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:49.676195 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:49.677534 (+ 1339us) server_negotiation.cc:407] Connection header received
0811 18:44:49.679463 (+ 1929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:49.679471 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:49.679594 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:49.679712 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:49.681239 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:49.681938 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:49.683451 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:49.683901 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:49.684037 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:49.687744 (+ 3707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:49.687752 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:49.687758 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:49.687805 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:49.690498 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:49.691463 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:49.691467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:49.691469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:49.691594 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:49.692121 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:49.692124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:49.692126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:49.692395 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:49.692508 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:49.693429 (+ 921us) server_negotiation.cc:299] Negotiation successful
0811 18:44:49.693609 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 18:44:49.695955 29725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:49.676494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.130:54279)
0811 18:44:49.677361 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:49.677412 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:49.679168 (+ 1756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:49.679954 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:49.679963 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:49.680482 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:44:49.681037 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:49.681053 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:49.682097 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:49.682107 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:49.683301 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:49.683312 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:49.684141 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:49.684149 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:49.684375 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:49.685106 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:44:49.685126 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:44:49.687488 (+ 2362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:44:49.690647 (+ 3159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:49.690659 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:49.690678 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:49.691340 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:49.691716 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:49.691724 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:49.691730 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:49.692011 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:49.692524 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:44:49.692535 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:44:49.693002 (+ 467us) client_negotiation.cc:769] Sending connection context
0811 18:44:49.693345 (+ 343us) client_negotiation.cc:240] Negotiation successful
0811 18:44:49.694386 (+ 1041us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":355,"threads_started":1}
I20260811 18:44:49.697600 29722 heartbeater.cc:347] Connected to a master server at 127.30.185.190:36929
I20260811 18:44:49.697958 29722 heartbeater.cc:464] Registering TS with master...
I20260811 18:44:49.698942 29722 heartbeater.cc:511] Master 127.30.185.190:36929 requested a full tablet report, sending...
I20260811 18:44:49.700696 29180 ts_manager.cc:194] Registered new tserver with Master: 9872f13b5ff546778e9baff77e240ce1 (127.30.185.130:40149)
I20260811 18:44:49.701694 29180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:54279
W20260811 18:44:50.034983 29726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:44:50.035286 29726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:44:50.035461 29726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:44:50.042785 29726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:44:50.042894 29726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:44:50.043095 29726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:44:50.053089 29726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:37527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=43589
--tserver_master_addrs=127.30.185.190:36929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:44:50.054487 29726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:44:50.055826 29726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:44:50.069226 29736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:50.069226 29738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:44:50.069250 29740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:44:50.070062 29726 server_base.cc:1034] running on GCE node
I20260811 18:44:50.070721 29726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:44:50.071679 29726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:44:50.072911 29726 hybrid_clock.cc:648] HybridClock initialized: now 1786473890072868 us; error 55 us; skew 500 ppm
I20260811 18:44:50.073298 29726 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:44:50 dist-test-slave-z8x9 krb5kdc[28129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473890, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:44:50.077531 29726 init.cc:377] Logged in from keytab as kudu/127.30.185.131@KRBTEST.COM (short username kudu)
I20260811 18:44:50.079068 29726 webserver.cc:460] Webserver started at http://127.30.185.131:43589/ using document root <none> and password file <none>
I20260811 18:44:50.079828 29726 fs_manager.cc:359] Metadata directory not provided
I20260811 18:44:50.079999 29726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:44:50.085672 29726 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:44:50.088514 29754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:50.090117 29726 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:44:50.090341 29726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "72943d582ff9476e805b085051c10f2a"
format_stamp: "Formatted at 2026-08-11 18:44:26 on dist-test-slave-z8x9"
server_key: "4d3e2b6b07c32ea95f0043890caa9588"
server_key_iv: "375bec15ed57f32ad65a57963e26b476"
server_key_version: "encryptionkey@0"
I20260811 18:44:50.091361 29726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:44:50.103605 29726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:44:50.108268 29726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:44:50.108569 29726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:44:50.109831 29726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:44:50.111760 29726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:44:50.111887 29726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:50.112008 29726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:44:50.112097 29726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:44:50.146899 29726 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:37527
I20260811 18:44:50.146957 29976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:37527 every 8 connection(s)
I20260811 18:44:50.148407 29726 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:44:50.151793 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 29726
Aug 11 18:44:50 dist-test-slave-z8x9 krb5kdc[28129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473890, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:44:50.172835 29499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:50.151202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:48835 (local address 127.30.185.190:36929)
0811 18:44:50.151447 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:50.151453 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:50.153065 (+ 1612us) server_negotiation.cc:407] Connection header received
0811 18:44:50.155250 (+ 2185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:50.155257 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:50.155368 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:50.155496 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:50.157302 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.158157 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.159909 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.160627 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.160789 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:50.164889 (+ 4100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:50.164895 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:50.164898 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:50.164933 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:50.169064 (+ 4131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:50.169981 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:50.169988 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:50.169992 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:50.170170 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:50.170745 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:50.170749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:50.170751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:50.171069 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:50.171257 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:50.172105 (+ 848us) server_negotiation.cc:299] Negotiation successful
0811 18:44:50.172377 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 18:44:50.174240 29980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:50.151881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.131:48835)
0811 18:44:50.152863 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:50.152921 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:50.154923 (+ 2002us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:50.155776 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:50.155786 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:50.156392 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:44:50.157046 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:50.157065 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.158353 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.158359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:50.159736 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:50.159750 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.160895 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.160904 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:50.161257 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:50.162092 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:44:50.162118 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:44:50.164651 (+ 2533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:44:50.169313 (+ 4662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:50.169321 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:50.169336 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:50.169819 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:50.170302 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:44:50.170307 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:44:50.170310 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:44:50.170599 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:44:50.171286 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:44:50.171296 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:44:50.171686 (+ 390us) client_negotiation.cc:769] Sending connection context
0811 18:44:50.171996 (+ 310us) client_negotiation.cc:240] Negotiation successful
0811 18:44:50.172635 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":411,"threads_started":1}
I20260811 18:44:50.175931 29978 heartbeater.cc:347] Connected to a master server at 127.30.185.190:36929
I20260811 18:44:50.176350 29978 heartbeater.cc:464] Registering TS with master...
I20260811 18:44:50.176975 29981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:50.152683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47074 (local address 127.30.185.190:36929)
0811 18:44:50.160415 (+ 7732us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:50.160422 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:50.160819 (+ 397us) server_negotiation.cc:407] Connection header received
0811 18:44:50.161022 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:50.161028 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:50.161100 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:50.161241 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:50.162863 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.163873 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.165670 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.166218 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.166322 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:50.170315 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:50.170323 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:50.170328 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:50.170365 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:50.173457 (+ 3092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:50.174557 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:50.174564 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:50.174567 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:50.174698 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:50.175227 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:50.175232 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:50.175235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:50.175579 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:50.175862 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:50.176275 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 18:44:50.176543 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7580,"thread_start_us":7432,"threads_started":1}
I20260811 18:44:50.177395 29978 heartbeater.cc:511] Master 127.30.185.190:36929 requested a full tablet report, sending...
I20260811 18:44:50.179333 29179 ts_manager.cc:194] Registered new tserver with Master: 72943d582ff9476e805b085051c10f2a (127.30.185.131:37527)
I20260811 18:44:50.180365 29179 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:48835
I20260811 18:44:50.192329 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:44:50.199456 29981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:50.193373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47080 (local address 127.30.185.190:36929)
0811 18:44:50.193544 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:50.193549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:50.194072 (+ 523us) server_negotiation.cc:407] Connection header received
0811 18:44:50.194357 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:50.194361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:50.194516 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:50.194673 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:44:50.195152 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.195900 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.197398 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.197816 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.197960 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:50.198503 (+ 543us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:44:50.198766 (+ 263us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 18:44:50.199160 (+ 394us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47080: token expired
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 18:44:50.199613 29981 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47080: token expired
W20260811 18:44:50.200119 29054 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.30.185.190:36929)
I20260811 18:44:50.214412 29981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:50.200906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47086 (local address 127.30.185.190:36929)
0811 18:44:50.201097 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:50.201103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:50.201118 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:44:50.201210 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:50.201215 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:50.201292 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:50.201393 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:50.202715 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.203607 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.205252 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.205681 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.205818 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:50.208694 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:50.208699 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:50.208703 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:50.208738 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:50.211344 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:50.212194 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:50.212205 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:50.212208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:50.212354 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:50.212741 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:50.212747 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:50.212750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:50.213110 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:50.213453 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:50.213805 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 18:44:50.214041 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 18:44:50.218257 29179 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:44:50.225369 29495 heartbeater.cc:503] Master 127.30.185.190:36929 was elected leader, sending a full tablet report...
W20260811 18:44:50.226058 29179 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:44:50.267541 29989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:50.239609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37078 (local address 127.30.185.129:38077)
0811 18:44:50.255096 (+ 15487us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:50.255107 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:50.255143 (+ 36us) server_negotiation.cc:407] Connection header received
0811 18:44:50.255492 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:50.255500 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:50.255760 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:50.255977 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:44:50.257232 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.259430 (+ 2198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.264461 (+ 5031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.266057 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.266337 (+ 280us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:50.266534 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:50.266999 (+ 465us) server_negotiation.cc:299] Negotiation successful
0811 18:44:50.267183 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15281,"thread_start_us":15552,"threads_started":1}
I20260811 18:44:50.268707 29990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:50.239613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:38077 (local address 127.0.0.1:37078)
0811 18:44:50.254744 (+ 15131us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:50.254816 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:50.255027 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:50.256303 (+ 1276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:50.256310 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:50.256351 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:44:50.256905 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:50.256924 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.259702 (+ 2778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.259709 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:50.264284 (+ 4575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:50.264294 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.266331 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.266337 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:50.266660 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:50.266733 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 18:44:50.268040 (+ 1307us) client_negotiation.cc:240] Negotiation successful
0811 18:44:50.268271 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14746,"mutex_wait_us":71,"thread_start_us":15657,"threads_started":1}
I20260811 18:44:50.269834 29991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:50.260192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:40149 (local address 127.0.0.1:43400)
0811 18:44:50.261918 (+ 1726us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:50.261936 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:50.262059 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:50.262866 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:50.262870 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:50.262902 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:44:50.263213 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:50.263223 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.264864 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.264871 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:50.266964 (+ 2093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:50.266979 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.268972 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.268978 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:50.269122 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:50.269146 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:44:50.269298 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 18:44:50.269444 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1581,"mutex_wait_us":4,"thread_start_us":1021,"threads_started":1}
I20260811 18:44:50.269888 29995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:50.256341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43400 (local address 127.30.185.130:40149)
0811 18:44:50.260736 (+ 4395us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:50.260743 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:50.262058 (+ 1315us) server_negotiation.cc:407] Connection header received
0811 18:44:50.262178 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:50.262185 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:50.262372 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:50.262541 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:44:50.263391 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.264697 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.267170 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.268482 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.268598 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:50.268726 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:50.269391 (+ 665us) server_negotiation.cc:299] Negotiation successful
0811 18:44:50.269577 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4236,"thread_start_us":4118,"threads_started":1}
I20260811 18:44:50.271159 29994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:50.240300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45726 (local address 127.30.185.131:37527)
0811 18:44:50.259662 (+ 19362us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:50.259676 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:50.261867 (+ 2191us) server_negotiation.cc:407] Connection header received
0811 18:44:50.261977 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:50.261983 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:50.262313 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:50.262542 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:44:50.264154 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.265509 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.268123 (+ 2614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.269726 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.269930 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:50.270112 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:50.270648 (+ 536us) server_negotiation.cc:299] Negotiation successful
0811 18:44:50.270820 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19108,"thread_start_us":19401,"threads_started":1}
I20260811 18:44:50.271097 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:50.239886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:37527 (local address 127.0.0.1:45726)
0811 18:44:50.261529 (+ 21643us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:50.261551 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:50.261696 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:50.262549 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:50.262555 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:50.262581 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:44:50.263936 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:50.263950 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.265654 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.265662 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:50.267932 (+ 2270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:50.267947 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.270304 (+ 2357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.270311 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:50.270452 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:50.270477 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:44:50.270590 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 18:44:50.270720 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21502,"mutex_wait_us":53,"thread_start_us":21698,"threads_started":1}
I20260811 18:44:50.273061 29380 tablet_service.cc:1467] Processing CreateTablet for tablet 65c507e10de645e79619759de55f506c (DEFAULT_TABLE table=test-table [id=59c8c5b1d54a49d48b2ac12100698680]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:44:50.274858 29380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65c507e10de645e79619759de55f506c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:44:50.274799 29853 tablet_service.cc:1467] Processing CreateTablet for tablet 65c507e10de645e79619759de55f506c (DEFAULT_TABLE table=test-table [id=59c8c5b1d54a49d48b2ac12100698680]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:44:50.275101 29604 tablet_service.cc:1467] Processing CreateTablet for tablet 65c507e10de645e79619759de55f506c (DEFAULT_TABLE table=test-table [id=59c8c5b1d54a49d48b2ac12100698680]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:44:50.276113 29853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65c507e10de645e79619759de55f506c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:44:50.276381 29604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65c507e10de645e79619759de55f506c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:44:50.288095 29999 tablet_bootstrap.cc:492] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a: Bootstrap starting.
I20260811 18:44:50.291059 30001 tablet_bootstrap.cc:492] T 65c507e10de645e79619759de55f506c P bd16c88da1cc42ab91697e45bd9e4bb3: Bootstrap starting.
I20260811 18:44:50.292276 29999 tablet_bootstrap.cc:654] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a: Neither blocks nor log segments found. Creating new log.
I20260811 18:44:50.292805 30002 tablet_bootstrap.cc:492] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1: Bootstrap starting.
I20260811 18:44:50.294296 29999 log.cc:824] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:44:50.295127 30001 tablet_bootstrap.cc:654] T 65c507e10de645e79619759de55f506c P bd16c88da1cc42ab91697e45bd9e4bb3: Neither blocks nor log segments found. Creating new log.
I20260811 18:44:50.296696 30002 tablet_bootstrap.cc:654] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1: Neither blocks nor log segments found. Creating new log.
I20260811 18:44:50.296957 30001 log.cc:824] T 65c507e10de645e79619759de55f506c P bd16c88da1cc42ab91697e45bd9e4bb3: Log is configured to *not* fsync() on all Append() calls
I20260811 18:44:50.297966 30002 log.cc:824] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1: Log is configured to *not* fsync() on all Append() calls
I20260811 18:44:50.301740 30001 tablet_bootstrap.cc:492] T 65c507e10de645e79619759de55f506c P bd16c88da1cc42ab91697e45bd9e4bb3: No bootstrap required, opened a new log
I20260811 18:44:50.302202 30001 ts_tablet_manager.cc:1397] T 65c507e10de645e79619759de55f506c P bd16c88da1cc42ab91697e45bd9e4bb3: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260811 18:44:50.304289 30002 tablet_bootstrap.cc:492] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1: No bootstrap required, opened a new log
I20260811 18:44:50.304608 30002 ts_tablet_manager.cc:1397] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.010s
I20260811 18:44:50.305328 29999 tablet_bootstrap.cc:492] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a: No bootstrap required, opened a new log
I20260811 18:44:50.305626 29999 ts_tablet_manager.cc:1397] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.005s
I20260811 18:44:50.312845 30002 raft_consensus.cc:348] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38077 } } peers { permanent_uuid: "9872f13b5ff546778e9baff77e240ce1" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40149 } } peers { permanent_uuid: "72943d582ff9476e805b085051c10f2a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37527 } }
I20260811 18:44:50.313390 30002 raft_consensus.cc:374] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:44:50.313529 30002 raft_consensus.cc:729] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9872f13b5ff546778e9baff77e240ce1, State: Initialized, Role: FOLLOWER
I20260811 18:44:50.313524 30001 raft_consensus.cc:348] T 65c507e10de645e79619759de55f506c P bd16c88da1cc42ab91697e45bd9e4bb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38077 } } peers { permanent_uuid: "9872f13b5ff546778e9baff77e240ce1" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40149 } } peers { permanent_uuid: "72943d582ff9476e805b085051c10f2a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37527 } }
I20260811 18:44:50.314085 30001 raft_consensus.cc:374] T 65c507e10de645e79619759de55f506c P bd16c88da1cc42ab91697e45bd9e4bb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:44:50.314208 30001 raft_consensus.cc:729] T 65c507e10de645e79619759de55f506c P bd16c88da1cc42ab91697e45bd9e4bb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd16c88da1cc42ab91697e45bd9e4bb3, State: Initialized, Role: FOLLOWER
I20260811 18:44:50.314225 30002 consensus_queue.cc:260] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38077 } } peers { permanent_uuid: "9872f13b5ff546778e9baff77e240ce1" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40149 } } peers { permanent_uuid: "72943d582ff9476e805b085051c10f2a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37527 } }
I20260811 18:44:50.314792 30001 consensus_queue.cc:260] T 65c507e10de645e79619759de55f506c P bd16c88da1cc42ab91697e45bd9e4bb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38077 } } peers { permanent_uuid: "9872f13b5ff546778e9baff77e240ce1" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40149 } } peers { permanent_uuid: "72943d582ff9476e805b085051c10f2a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37527 } }
I20260811 18:44:50.315653 29722 heartbeater.cc:503] Master 127.30.185.190:36929 was elected leader, sending a full tablet report...
I20260811 18:44:50.316162 30002 ts_tablet_manager.cc:1428] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s
I20260811 18:44:50.318748 30001 ts_tablet_manager.cc:1428] T 65c507e10de645e79619759de55f506c P bd16c88da1cc42ab91697e45bd9e4bb3: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260811 18:44:50.318812 29999 raft_consensus.cc:348] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38077 } } peers { permanent_uuid: "9872f13b5ff546778e9baff77e240ce1" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40149 } } peers { permanent_uuid: "72943d582ff9476e805b085051c10f2a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37527 } }
I20260811 18:44:50.319263 29999 raft_consensus.cc:374] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:44:50.319357 29999 raft_consensus.cc:729] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72943d582ff9476e805b085051c10f2a, State: Initialized, Role: FOLLOWER
I20260811 18:44:50.319963 29999 consensus_queue.cc:260] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38077 } } peers { permanent_uuid: "9872f13b5ff546778e9baff77e240ce1" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40149 } } peers { permanent_uuid: "72943d582ff9476e805b085051c10f2a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37527 } }
I20260811 18:44:50.321161 29978 heartbeater.cc:503] Master 127.30.185.190:36929 was elected leader, sending a full tablet report...
I20260811 18:44:50.321756 29999 ts_tablet_manager.cc:1428] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a: Time spent starting tablet: real 0.016s user 0.009s sys 0.005s
W20260811 18:44:50.400156 29979 tablet.cc:2369] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:44:50.411831 30014 raft_consensus.cc:482] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:44:50.412134 30014 raft_consensus.cc:504] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38077 } } peers { permanent_uuid: "9872f13b5ff546778e9baff77e240ce1" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40149 } } peers { permanent_uuid: "72943d582ff9476e805b085051c10f2a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37527 } }
I20260811 18:44:50.414067 30014 leader_election.cc:288] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd16c88da1cc42ab91697e45bd9e4bb3 (127.30.185.129:38077), 72943d582ff9476e805b085051c10f2a (127.30.185.131:37527)
I20260811 18:44:50.416539 30020 raft_consensus.cc:482] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:44:50.416868 30020 raft_consensus.cc:504] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38077 } } peers { permanent_uuid: "9872f13b5ff546778e9baff77e240ce1" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40149 } } peers { permanent_uuid: "72943d582ff9476e805b085051c10f2a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37527 } }
I20260811 18:44:50.419176 30020 leader_election.cc:288] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd16c88da1cc42ab91697e45bd9e4bb3 (127.30.185.129:38077), 9872f13b5ff546778e9baff77e240ce1 (127.30.185.130:40149)
I20260811 18:44:50.423331 29989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:50.414686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:51681 (local address 127.30.185.129:38077)
0811 18:44:50.414891 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:50.414898 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:50.415847 (+ 949us) server_negotiation.cc:407] Connection header received
0811 18:44:50.416018 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:50.416023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:50.416101 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:50.416205 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:44:50.417112 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.417993 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.420630 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.422015 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.422151 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:50.422234 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:50.422918 (+ 684us) server_negotiation.cc:299] Negotiation successful
0811 18:44:50.423074 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 18:44:50.423442 29994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:50.414671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:45985 (local address 127.30.185.131:37527)
0811 18:44:50.414892 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:50.414898 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:50.415776 (+ 878us) server_negotiation.cc:407] Connection header received
0811 18:44:50.416014 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:50.416020 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:50.416101 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:50.416205 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:44:50.417221 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.418354 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.420929 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.422258 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.422402 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:50.422473 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:50.423059 (+ 586us) server_negotiation.cc:299] Negotiation successful
0811 18:44:50.423183 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 18:44:50.424165 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:50.414396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:37527 (local address 127.30.185.130:45985)
0811 18:44:50.415635 (+ 1239us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:50.415658 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:50.415843 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:50.416418 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:50.416422 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:50.416457 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:44:50.417005 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:50.417036 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.418496 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.418501 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:50.420744 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:50.420759 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.422654 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.422659 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:50.422895 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:50.422924 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 18:44:50.423591 (+ 667us) client_negotiation.cc:240] Negotiation successful
0811 18:44:50.423725 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1087,"mutex_wait_us":37,"thread_start_us":761,"threads_started":1}
I20260811 18:44:50.425348 30024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:50.414520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:38077 (local address 127.30.185.130:51681)
0811 18:44:50.415709 (+ 1189us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:50.415726 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:50.415883 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:50.416252 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:50.416257 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:50.416283 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:44:50.416903 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:50.416920 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.418151 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.418157 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:50.420416 (+ 2259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:50.420432 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.422439 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.422446 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:50.422642 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:50.422740 (+ 98us) client_negotiation.cc:769] Sending connection context
0811 18:44:50.424763 (+ 2023us) client_negotiation.cc:240] Negotiation successful
0811 18:44:50.424922 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1050,"mutex_wait_us":51,"spinlock_wait_cycles":55936,"thread_start_us":640,"threads_started":1}
I20260811 18:44:50.425402 29900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65c507e10de645e79619759de55f506c" candidate_uuid: "9872f13b5ff546778e9baff77e240ce1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72943d582ff9476e805b085051c10f2a" is_pre_election: true
I20260811 18:44:50.425981 29900 raft_consensus.cc:2454] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9872f13b5ff546778e9baff77e240ce1 in term 0.
W20260811 18:44:50.426337 29724 tablet.cc:2369] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:44:50.426661 29420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65c507e10de645e79619759de55f506c" candidate_uuid: "9872f13b5ff546778e9baff77e240ce1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" is_pre_election: true
I20260811 18:44:50.427045 29523 leader_election.cc:302] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72943d582ff9476e805b085051c10f2a, 9872f13b5ff546778e9baff77e240ce1; no voters:
I20260811 18:44:50.427258 29420 raft_consensus.cc:2454] T 65c507e10de645e79619759de55f506c P bd16c88da1cc42ab91697e45bd9e4bb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9872f13b5ff546778e9baff77e240ce1 in term 0.
I20260811 18:44:50.427520 30014 raft_consensus.cc:2781] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:44:50.427670 30014 raft_consensus.cc:482] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:44:50.427776 30014 raft_consensus.cc:3037] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:44:50.431097 30014 raft_consensus.cc:504] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38077 } } peers { permanent_uuid: "9872f13b5ff546778e9baff77e240ce1" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40149 } } peers { permanent_uuid: "72943d582ff9476e805b085051c10f2a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37527 } }
I20260811 18:44:50.432358 29420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65c507e10de645e79619759de55f506c" candidate_uuid: "9872f13b5ff546778e9baff77e240ce1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3"
I20260811 18:44:50.432579 29420 raft_consensus.cc:3037] T 65c507e10de645e79619759de55f506c P bd16c88da1cc42ab91697e45bd9e4bb3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:44:50.432973 29980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:50.419533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:38077 (local address 127.30.185.131:60051)
0811 18:44:50.419735 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:50.419755 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:50.419877 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:50.424370 (+ 4493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:50.424376 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:50.424407 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:44:50.424603 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:50.424612 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.427702 (+ 3090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.427708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:50.429723 (+ 2015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:50.429741 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.432143 (+ 2402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.432148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:50.432389 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:50.432423 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 18:44:50.432519 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 18:44:50.432670 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 18:44:50.435994 29989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:50.420239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:60051 (local address 127.30.185.129:38077)
0811 18:44:50.423564 (+ 3325us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:50.423571 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:50.423590 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:44:50.423644 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:50.423649 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:50.423715 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:50.423815 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:44:50.424750 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.425943 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.430482 (+ 4539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.431792 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.431977 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:50.432054 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:50.435518 (+ 3464us) server_negotiation.cc:299] Negotiation successful
0811 18:44:50.435661 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3226,"thread_start_us":15136,"threads_started":1}
I20260811 18:44:50.436237 30014 leader_election.cc:288] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1 [CANDIDATE]: Term 1 election: Requested vote from peers bd16c88da1cc42ab91697e45bd9e4bb3 (127.30.185.129:38077), 72943d582ff9476e805b085051c10f2a (127.30.185.131:37527)
I20260811 18:44:50.436353 29900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65c507e10de645e79619759de55f506c" candidate_uuid: "9872f13b5ff546778e9baff77e240ce1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72943d582ff9476e805b085051c10f2a"
I20260811 18:44:50.436609 29900 raft_consensus.cc:3037] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:44:50.436834 29420 raft_consensus.cc:2454] T 65c507e10de645e79619759de55f506c P bd16c88da1cc42ab91697e45bd9e4bb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9872f13b5ff546778e9baff77e240ce1 in term 1.
I20260811 18:44:50.437335 29522 leader_election.cc:302] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9872f13b5ff546778e9baff77e240ce1, bd16c88da1cc42ab91697e45bd9e4bb3; no voters:
I20260811 18:44:50.437695 30014 raft_consensus.cc:2781] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:44:50.437577 29418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65c507e10de645e79619759de55f506c" candidate_uuid: "72943d582ff9476e805b085051c10f2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" is_pre_election: true
I20260811 18:44:50.438047 29418 raft_consensus.cc:2379] T 65c507e10de645e79619759de55f506c P bd16c88da1cc42ab91697e45bd9e4bb3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 72943d582ff9476e805b085051c10f2a in current term 1: Already voted for candidate 9872f13b5ff546778e9baff77e240ce1 in this term.
I20260811 18:44:50.438265 30014 raft_consensus.cc:686] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1 [term 1 LEADER]: Becoming Leader. State: Replica: 9872f13b5ff546778e9baff77e240ce1, State: Running, Role: LEADER
I20260811 18:44:50.438831 30014 consensus_queue.cc:237] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38077 } } peers { permanent_uuid: "9872f13b5ff546778e9baff77e240ce1" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40149 } } peers { permanent_uuid: "72943d582ff9476e805b085051c10f2a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37527 } }
W20260811 18:44:50.440465 29498 tablet.cc:2369] T 65c507e10de645e79619759de55f506c P bd16c88da1cc42ab91697e45bd9e4bb3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:44:50.440128 29900 raft_consensus.cc:2454] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9872f13b5ff546778e9baff77e240ce1 in term 1.
I20260811 18:44:50.442127 29980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:50.419963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:40149 (local address 127.30.185.131:50507)
0811 18:44:50.433219 (+ 13256us) negotiation.cc:107] Waiting for socket to connect
0811 18:44:50.433239 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:44:50.433413 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:44:50.433932 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:44:50.433936 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:44:50.433963 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:44:50.434163 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:50.434175 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.435523 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.435527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:50.439636 (+ 4109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:44:50.439675 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.441196 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.441203 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:44:50.441510 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:50.441554 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 18:44:50.441644 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 18:44:50.441789 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13121,"thread_start_us":15181,"threads_started":1}
I20260811 18:44:50.442826 29995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:50.420457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:50507 (local address 127.30.185.130:40149)
0811 18:44:50.420645 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:50.420652 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:50.433316 (+ 12664us) server_negotiation.cc:407] Connection header received
0811 18:44:50.433552 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:50.433559 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:50.433645 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:50.433707 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:44:50.434326 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.435347 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.439800 (+ 4453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.440989 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.441168 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:50.441262 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:50.442349 (+ 1087us) server_negotiation.cc:299] Negotiation successful
0811 18:44:50.442490 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 18:44:50.443873 29639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65c507e10de645e79619759de55f506c" candidate_uuid: "72943d582ff9476e805b085051c10f2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9872f13b5ff546778e9baff77e240ce1" is_pre_election: true
I20260811 18:44:50.445029 29760 leader_election.cc:302] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72943d582ff9476e805b085051c10f2a; no voters: 9872f13b5ff546778e9baff77e240ce1, bd16c88da1cc42ab91697e45bd9e4bb3
I20260811 18:44:50.445525 30020 raft_consensus.cc:2726] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:44:50.445222 29180 catalog_manager.cc:5495] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9872f13b5ff546778e9baff77e240ce1 (127.30.185.130). New cstate: current_term: 1 leader_uuid: "9872f13b5ff546778e9baff77e240ce1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9872f13b5ff546778e9baff77e240ce1" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 40149 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72943d582ff9476e805b085051c10f2a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37527 } health_report { overall_health: UNKNOWN } } }
I20260811 18:44:50.496897 29995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:50.491077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43414 (local address 127.30.185.130:40149)
0811 18:44:50.491270 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:50.491276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:50.491375 (+ 99us) server_negotiation.cc:407] Connection header received
0811 18:44:50.491649 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:50.491653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:50.491731 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:50.491909 (+ 178us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:44:50.492284 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.493050 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.494552 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.495038 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.495177 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:50.495695 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:44:50.496286 (+ 591us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:44:50.496446 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:50.496514 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 18:44:50.496598 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 18:44:50.505528 29418 raft_consensus.cc:1261] T 65c507e10de645e79619759de55f506c P bd16c88da1cc42ab91697e45bd9e4bb3 [term 1 FOLLOWER]: Refusing update from remote peer 9872f13b5ff546778e9baff77e240ce1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:44:50.505579 29900 raft_consensus.cc:1261] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a [term 1 FOLLOWER]: Refusing update from remote peer 9872f13b5ff546778e9baff77e240ce1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:44:50.506501 30030 consensus_queue.cc:1035] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd16c88da1cc42ab91697e45bd9e4bb3" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 38077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:44:50.506902 30014 consensus_queue.cc:1035] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72943d582ff9476e805b085051c10f2a" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 37527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:44:50.523583 30041 mvcc.cc:205] Tried to move back new op lower bound from 7317397055500034048 to 7317397055242199040. Current Snapshot: MvccSnapshot[applied={T|T < 7317397055500034048}]
I20260811 18:44:50.533667 29179 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:44:50.533981 29179 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:44:50.537892 29179 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212: Sending DeleteTablet for 3 replicas of tablet 65c507e10de645e79619759de55f506c
I20260811 18:44:50.539276 29604 tablet_service.cc:1514] Processing DeleteTablet for tablet 65c507e10de645e79619759de55f506c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:44:50 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:43400
I20260811 18:44:50.539265 29380 tablet_service.cc:1514] Processing DeleteTablet for tablet 65c507e10de645e79619759de55f506c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:44:50 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:37078
I20260811 18:44:50.539762 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 29246
I20260811 18:44:50.540076 29853 tablet_service.cc:1514] Processing DeleteTablet for tablet 65c507e10de645e79619759de55f506c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:44:50 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:45726
I20260811 18:44:50.550601 30058 tablet_replica.cc:330] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1: stopping tablet replica
I20260811 18:44:50.551227 30058 raft_consensus.cc:2229] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:44:50.552011 30058 raft_consensus.cc:2258] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:44:50.553056 30056 tablet_replica.cc:330] T 65c507e10de645e79619759de55f506c P bd16c88da1cc42ab91697e45bd9e4bb3: stopping tablet replica
I20260811 18:44:50.553543 30056 raft_consensus.cc:2229] T 65c507e10de645e79619759de55f506c P bd16c88da1cc42ab91697e45bd9e4bb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:44:50.554104 30056 raft_consensus.cc:2258] T 65c507e10de645e79619759de55f506c P bd16c88da1cc42ab91697e45bd9e4bb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:44:50.554909 30058 ts_tablet_manager.cc:1905] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:44:50.556322 30056 ts_tablet_manager.cc:1905] T 65c507e10de645e79619759de55f506c P bd16c88da1cc42ab91697e45bd9e4bb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:44:50.563498 30058 ts_tablet_manager.cc:1918] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:44:50.563745 30058 log.cc:1196] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/65c507e10de645e79619759de55f506c
I20260811 18:44:50.563738 30056 ts_tablet_manager.cc:1918] T 65c507e10de645e79619759de55f506c P bd16c88da1cc42ab91697e45bd9e4bb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:44:50.563962 30056 log.cc:1196] T 65c507e10de645e79619759de55f506c P bd16c88da1cc42ab91697e45bd9e4bb3: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/wal/wals/65c507e10de645e79619759de55f506c
I20260811 18:44:50.564532 30058 ts_tablet_manager.cc:1939] T 65c507e10de645e79619759de55f506c P 9872f13b5ff546778e9baff77e240ce1: Deleting consensus metadata
I20260811 18:44:50.564636 30056 ts_tablet_manager.cc:1939] T 65c507e10de645e79619759de55f506c P bd16c88da1cc42ab91697e45bd9e4bb3: Deleting consensus metadata
I20260811 18:44:50.565213 30059 tablet_replica.cc:330] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a: stopping tablet replica
I20260811 18:44:50.566766 29161 catalog_manager.cc:4841] TS bd16c88da1cc42ab91697e45bd9e4bb3 (127.30.185.129:38077): tablet 65c507e10de645e79619759de55f506c (table test-table [id=59c8c5b1d54a49d48b2ac12100698680]) successfully deleted
I20260811 18:44:50.566937 29989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:50.541738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37094 (local address 127.30.185.129:38077)
0811 18:44:50.541895 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:50.541900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:50.541916 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:44:50.542002 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:50.542010 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:50.542123 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:50.542234 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:50.544524 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.546084 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.548809 (+ 2725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.549861 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.550015 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:50.556887 (+ 6872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:50.556902 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:50.556918 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:50.556960 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:50.561362 (+ 4402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:50.562383 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:50.562392 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:50.562396 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:50.562566 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:50.563911 (+ 1345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:50.563917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:50.563923 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:50.564462 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:50.564822 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:50.566327 (+ 1505us) server_negotiation.cc:299] Negotiation successful
0811 18:44:50.566576 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:44:50.567139 29161 catalog_manager.cc:4841] TS 9872f13b5ff546778e9baff77e240ce1 (127.30.185.130:40149): tablet 65c507e10de645e79619759de55f506c (table test-table [id=59c8c5b1d54a49d48b2ac12100698680]) successfully deleted
I20260811 18:44:50.567607 29488 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:44:50.567767 30059 raft_consensus.cc:2229] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:44:50.568298 30059 raft_consensus.cc:2258] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:44:50.573503 30059 ts_tablet_manager.cc:1905] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:44:50.581106 30059 ts_tablet_manager.cc:1918] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:44:50.581383 30059 log.cc:1196] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/65c507e10de645e79619759de55f506c
I20260811 18:44:50.581998 30059 ts_tablet_manager.cc:1939] T 65c507e10de645e79619759de55f506c P 72943d582ff9476e805b085051c10f2a: Deleting consensus metadata
I20260811 18:44:50.584234 29163 catalog_manager.cc:4841] TS 72943d582ff9476e805b085051c10f2a (127.30.185.131:37527): tablet 65c507e10de645e79619759de55f506c (table test-table [id=59c8c5b1d54a49d48b2ac12100698680]) successfully deleted
I20260811 18:44:50.670601 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 29246
I20260811 18:44:50.704113 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 29502
I20260811 18:44:50.723810 29995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:50.705068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43430 (local address 127.30.185.130:40149)
0811 18:44:50.705292 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:50.705299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:50.705355 (+ 56us) server_negotiation.cc:407] Connection header received
0811 18:44:50.705578 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:50.705582 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:50.705654 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:50.705761 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:50.707144 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.707872 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.709318 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.709793 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.709946 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:50.717413 (+ 7467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:50.717429 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:50.717445 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:50.717489 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:50.720731 (+ 3242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:50.721591 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:50.721609 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:50.721611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:50.721745 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:50.722179 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:50.722184 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:50.722186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:50.722597 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:50.722949 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:50.723225 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 18:44:50.723466 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 18:44:50.724756 29713 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:44:50.840628 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 29502
I20260811 18:44:50.872673 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 29726
I20260811 18:44:50.891443 29994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:44:50.873540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45738 (local address 127.30.185.131:37527)
0811 18:44:50.873730 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 18:44:50.873735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:44:50.873754 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:44:50.873897 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:44:50.873901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:44:50.873964 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:44:50.874052 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:44:50.875373 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.876091 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.877460 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:44:50.877852 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:44:50.877961 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:44:50.885718 (+ 7757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:44:50.885737 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:44:50.885753 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:44:50.885792 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:44:50.888706 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:50.889437 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:50.889445 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:50.889447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:50.889589 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:44:50.889921 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:44:50.889925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:44:50.889927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:44:50.890300 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:44:50.890515 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 18:44:50.890821 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 18:44:50.891049 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 18:44:50.892197 29967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:44:50.982479 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 29726
I20260811 18:44:51.005676 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 29130
I20260811 18:44:51.006822 29237 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:44:51.124614 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 29130
I20260811 18:45:11.157902 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:36929
--webserver_interface=127.30.185.190
--webserver_port=42741
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:36929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:45:11.513834 30068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:11.514302 30068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:11.514488 30068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:11.522163 30068 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 18:45:11.522315 30068 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 18:45:11.522389 30068 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 18:45:11.522491 30068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:45:11.522578 30068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:11.522711 30068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:11.522804 30068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:45:11.522897 30068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:45:11.533161 30068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:36929
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:36929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=42741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:11.534605 30068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:11.536036 30068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:11.548944 30083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:11.548918 30079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:11.548877 30078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:11.549819 30068 server_base.cc:1034] running on GCE node
I20260811 18:45:11.550626 30068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:11.552119 30068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:11.553378 30068 hybrid_clock.cc:648] HybridClock initialized: now 1786473911553303 us; error 50 us; skew 500 ppm
I20260811 18:45:11.553727 30068 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:45:11 dist-test-slave-z8x9 krb5kdc[28129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473911, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:45:11.560171 30068 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:45:11.561827 30068 webserver.cc:460] Webserver started at http://127.30.185.190:42741/ using document root <none> and password file <none>
I20260811 18:45:11.562570 30068 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:11.562711 30068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:11.568361 30068 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 18:45:11.571331 30091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:11.572695 30068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:45:11.572865 30068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "5cd1365a8a1047d1bec51d03529ad212"
format_stamp: "Formatted at 2026-08-11 18:44:25 on dist-test-slave-z8x9"
server_key: "4a5725a31e5b88d8e5f411e7dc921749"
server_key_iv: "faa9a9d0311eef54f135095635648f82"
server_key_version: "encryptionkey@0"
I20260811 18:45:11.573873 30068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:11.591135 30068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:11.595397 30068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:11.595660 30068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:11.615625 30068 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:36929
I20260811 18:45:11.615658 30170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:36929 every 8 connection(s)
I20260811 18:45:11.617236 30068 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:45:11.622954 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 30068
I20260811 18:45:11.624087 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:38077
--local_ip_for_outbound_sockets=127.30.185.129
--tserver_master_addrs=127.30.185.190:36929
--webserver_port=45859
--webserver_interface=127.30.185.129
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:45:11.625136 30171 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:45:11.630568 30171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212: Bootstrap starting.
I20260811 18:45:11.653649 30171 log.cc:824] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:11.671260 30171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:45:11.671886 30171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212: Bootstrap complete.
I20260811 18:45:11.682435 30171 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd1365a8a1047d1bec51d03529ad212" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 36929 } }
I20260811 18:45:11.683334 30171 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd1365a8a1047d1bec51d03529ad212, State: Initialized, Role: FOLLOWER
I20260811 18:45:11.684065 30171 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd1365a8a1047d1bec51d03529ad212" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 36929 } }
I20260811 18:45:11.684325 30171 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:45:11.684445 30171 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:45:11.684646 30171 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 2 FOLLOWER]: Advancing to term 3
I20260811 18:45:11.688652 30171 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd1365a8a1047d1bec51d03529ad212" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 36929 } }
I20260811 18:45:11.689230 30171 leader_election.cc:302] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cd1365a8a1047d1bec51d03529ad212; no voters:
I20260811 18:45:11.690231 30171 leader_election.cc:288] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 18:45:11.690424 30180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 18:45:11.691613 30180 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [term 3 LEADER]: Becoming Leader. State: Replica: 5cd1365a8a1047d1bec51d03529ad212, State: Running, Role: LEADER
I20260811 18:45:11.692233 30180 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd1365a8a1047d1bec51d03529ad212" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 36929 } }
I20260811 18:45:11.692663 30171 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:45:11.694839 30183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cd1365a8a1047d1bec51d03529ad212. Latest consensus state: current_term: 3 leader_uuid: "5cd1365a8a1047d1bec51d03529ad212" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd1365a8a1047d1bec51d03529ad212" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 36929 } } }
I20260811 18:45:11.695240 30183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [sys.catalog]: This master's current role is: LEADER
I20260811 18:45:11.694948 30182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "5cd1365a8a1047d1bec51d03529ad212" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cd1365a8a1047d1bec51d03529ad212" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 36929 } } }
I20260811 18:45:11.695374 30182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212 [sys.catalog]: This master's current role is: LEADER
I20260811 18:45:11.696226 30188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:45:11.717723 30188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:45:11.719738 30188 catalog_manager.cc:1259] Loaded cluster ID: e59b1719777b4493b26dbd931f8edc00
I20260811 18:45:11.719897 30188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:45:11.725138 30188 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:45:11.727451 30188 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212: Loaded TSK: 0
I20260811 18:45:11.727655 30188 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5cd1365a8a1047d1bec51d03529ad212: Loaded TSK: 1
F20260811 18:45:11.729139 30188 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f651ba4b9ad google::LogMessage::Fail() at ??:0
@ 0x7f651ba4e88a google::LogMessage::SendToLog() at ??:0
@ 0x7f651ba4b4de google::LogMessage::Flush() at ??:0
@ 0x7f651ba4ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f651f7af7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f651f79e73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f653147a1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f6531479783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f65315052a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f65315050c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f653154f5e9 std::function<>::operator()() at ??:0
@ 0x7f653147dcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f653147bc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6531503610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f6531503439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f651e697345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f651e69a91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f651e69a799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f651e675eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f652194f6db start_thread at ??:0
@ 0x7f651a8e561f clone at ??:0
W20260811 18:45:12.054388 30176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:12.054672 30176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:12.054834 30176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:12.062278 30176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:12.062389 30176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:12.062601 30176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:45:12.072532 30176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:38077
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=45859
--tserver_master_addrs=127.30.185.190:36929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:12.073868 30176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:12.075199 30176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:12.089325 30263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:12.089315 30257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:12.089370 30258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:12.090152 30176 server_base.cc:1034] running on GCE node
I20260811 18:45:12.090813 30176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:12.091853 30176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:12.093063 30176 hybrid_clock.cc:648] HybridClock initialized: now 1786473912093019 us; error 40 us; skew 500 ppm
I20260811 18:45:12.093433 30176 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:45:12 dist-test-slave-z8x9 krb5kdc[28129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473912, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:45:12.096698 30176 init.cc:377] Logged in from keytab as kudu/127.30.185.129@KRBTEST.COM (short username kudu)
I20260811 18:45:12.098361 30176 webserver.cc:460] Webserver started at http://127.30.185.129:45859/ using document root <none> and password file <none>
I20260811 18:45:12.099135 30176 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:12.099310 30176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:12.105115 30176 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 18:45:12.108498 30281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:12.109995 30176 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:45:12.110183 30176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "bd16c88da1cc42ab91697e45bd9e4bb3"
format_stamp: "Formatted at 2026-08-11 18:44:25 on dist-test-slave-z8x9"
server_key: "62ffffcdc8d47bea24c06b920a4a4db6"
server_key_iv: "f000db7f3ac4d793f4ccccd430906808"
server_key_version: "encryptionkey@0"
I20260811 18:45:12.111218 30176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:12.140946 30176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:12.145561 30176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:12.145871 30176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:12.146994 30176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:45:12.148437 30176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:45:12.148567 30176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:12.148710 30176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:45:12.148806 30176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:12.188215 30176 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:38077
I20260811 18:45:12.188320 30480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:38077 every 8 connection(s)
I20260811 18:45:12.189735 30176 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:45:12.190948 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 30176
I20260811 18:45:12.191812 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:40149
--local_ip_for_outbound_sockets=127.30.185.130
--tserver_master_addrs=127.30.185.190:36929
--webserver_port=34273
--webserver_interface=127.30.185.130
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:45:12.195345 30484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:12.193410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.129:36341)
0811 18:45:12.194234 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:12.194496 (+ 262us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":351,"threads_started":1}
W20260811 18:45:12.195981 30481 heartbeater.cc:644] Failed to heartbeat to 127.30.185.190:36929 (0 consecutive failures): Network error: Failed to ping master at 127.30.185.190:36929: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
I20260811 18:45:12.196918 30484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:12.196585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.129:51589)
0811 18:45:12.196738 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:12.196806 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 18:45:12.197897 30484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:12.197561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.129:42959)
0811 18:45:12.197689 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:12.197778 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 18:45:12.198190 30481 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:45:12.541458 30486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:12.541754 30486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:12.541850 30486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:12.549418 30486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:12.549542 30486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:12.549659 30486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:45:12.559754 30486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:40149
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=34273
--tserver_master_addrs=127.30.185.190:36929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:12.561048 30486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:12.562412 30486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:12.576059 30494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:12.576112 30501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:12.576028 30496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:12.576896 30486 server_base.cc:1034] running on GCE node
I20260811 18:45:12.577598 30486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:12.578500 30486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:12.579742 30486 hybrid_clock.cc:648] HybridClock initialized: now 1786473912579688 us; error 43 us; skew 500 ppm
I20260811 18:45:12.580070 30486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:45:12 dist-test-slave-z8x9 krb5kdc[28129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473912, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:45:12.583627 30486 init.cc:377] Logged in from keytab as kudu/127.30.185.130@KRBTEST.COM (short username kudu)
I20260811 18:45:12.585319 30486 webserver.cc:460] Webserver started at http://127.30.185.130:34273/ using document root <none> and password file <none>
I20260811 18:45:12.586082 30486 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:12.586223 30486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:12.591830 30486 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 18:45:12.594862 30512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:12.596313 30486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:45:12.596558 30486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "9872f13b5ff546778e9baff77e240ce1"
format_stamp: "Formatted at 2026-08-11 18:44:26 on dist-test-slave-z8x9"
server_key: "0d9d3310df96a5e5db55569df098236d"
server_key_iv: "e5a4a9d372f8b188f2e567b6ada93882"
server_key_version: "encryptionkey@0"
I20260811 18:45:12.597852 30486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:12.617944 30486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:12.622232 30486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:12.622535 30486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:12.623574 30486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:45:12.624970 30486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:45:12.625097 30486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:12.625267 30486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:45:12.625356 30486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:12.661106 30486 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:40149
I20260811 18:45:12.661252 30729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:40149 every 8 connection(s)
I20260811 18:45:12.663149 30486 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:45:12.666548 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 30486
I20260811 18:45:12.667374 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:37527
--local_ip_for_outbound_sockets=127.30.185.131
--tserver_master_addrs=127.30.185.190:36929
--webserver_port=43589
--webserver_interface=127.30.185.131
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:45:12.669998 30732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:12.667138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.130:52609)
0811 18:45:12.668329 (+ 1191us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:12.668692 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":821,"thread_start_us":582,"threads_started":1}
W20260811 18:45:12.671051 30730 heartbeater.cc:644] Failed to heartbeat to 127.30.185.190:36929 (0 consecutive failures): Network error: Failed to ping master at 127.30.185.190:36929: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
I20260811 18:45:12.672230 30732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:12.671694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.130:56817)
0811 18:45:12.671936 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:12.672047 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 18:45:12.673485 30732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:12.672922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.130:42583)
0811 18:45:12.673154 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:12.673302 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 18:45:12.673846 30730 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:45:13.016234 30737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:13.016548 30737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:13.016644 30737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:13.024133 30737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:13.024300 30737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:13.024420 30737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:45:13.034410 30737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:37527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=43589
--tserver_master_addrs=127.30.185.190:36929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:13.035732 30737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:13.037005 30737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:13.050940 30747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:13.051016 30751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:13.050981 30748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:13.052156 30737 server_base.cc:1034] running on GCE node
I20260811 18:45:13.052790 30737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:13.053810 30737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:13.055045 30737 hybrid_clock.cc:648] HybridClock initialized: now 1786473913054989 us; error 53 us; skew 500 ppm
I20260811 18:45:13.055374 30737 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:45:13 dist-test-slave-z8x9 krb5kdc[28129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473913, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:45:13.058696 30737 init.cc:377] Logged in from keytab as kudu/127.30.185.131@KRBTEST.COM (short username kudu)
I20260811 18:45:13.060269 30737 webserver.cc:460] Webserver started at http://127.30.185.131:43589/ using document root <none> and password file <none>
I20260811 18:45:13.060986 30737 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:13.061142 30737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:13.066790 30737 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 18:45:13.069747 30763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:13.071695 30737 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 18:45:13.071907 30737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "72943d582ff9476e805b085051c10f2a"
format_stamp: "Formatted at 2026-08-11 18:44:26 on dist-test-slave-z8x9"
server_key: "4d3e2b6b07c32ea95f0043890caa9588"
server_key_iv: "375bec15ed57f32ad65a57963e26b476"
server_key_version: "encryptionkey@0"
I20260811 18:45:13.072938 30737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:13.084909 30737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:13.089414 30737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:13.089712 30737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:13.090763 30737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:45:13.092217 30737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:45:13.092346 30737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:13.092480 30737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:45:13.092568 30737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:13.128712 30737 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:37527
I20260811 18:45:13.128813 30970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:37527 every 8 connection(s)
I20260811 18:45:13.130268 30737 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:45:13.133114 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 30737
I20260811 18:45:13.135880 30974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:13.133633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.131:38361)
0811 18:45:13.134547 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:13.134864 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":393,"threads_started":1}
W20260811 18:45:13.136548 30971 heartbeater.cc:644] Failed to heartbeat to 127.30.185.190:36929 (0 consecutive failures): Network error: Failed to ping master at 127.30.185.190:36929: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
I20260811 18:45:13.137508 30974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:13.137105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.131:39863)
0811 18:45:13.137312 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:13.137398 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 18:45:13.138438 30974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:13.138106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.131:55245)
0811 18:45:13.138250 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:13.138327 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 18:45:13.138738 30971 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 18:45:13.200155 30982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:13.199169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.129:51437)
0811 18:45:13.199802 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:13.199903 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":364,"threads_started":1}
I20260811 18:45:13.675334 30983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:13.674451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.130:40237)
0811 18:45:13.674957 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:13.675065 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":259,"threads_started":1}
I20260811 18:45:14.140048 30984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:14.139277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.131:46733)
0811 18:45:14.139730 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:14.139808 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":232,"threads_started":1}
I20260811 18:45:14.202095 30985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:14.201208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.129:42843)
0811 18:45:14.201704 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:14.201795 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":230,"threads_started":1}
I20260811 18:45:14.677142 30986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:14.676299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.130:39567)
0811 18:45:14.676847 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:14.676950 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":259,"threads_started":1}
I20260811 18:45:15.141932 30987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:15.141213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.131:42221)
0811 18:45:15.141672 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:15.141751 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":231,"threads_started":1}
I20260811 18:45:15.203948 30988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:15.203225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.129:44951)
0811 18:45:15.203669 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:15.203768 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":232,"threads_started":1}
I20260811 18:45:15.678934 30991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:15.678145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.130:38369)
0811 18:45:15.678650 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:15.678754 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":270,"threads_started":1}
I20260811 18:45:16.143604 30992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:16.142885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.131:32901)
0811 18:45:16.143343 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:16.143426 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":245,"threads_started":1}
I20260811 18:45:16.205770 30996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:16.204990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.129:41719)
0811 18:45:16.205503 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:16.205586 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":305,"threads_started":1}
I20260811 18:45:16.680601 30998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:16.679869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.130:38929)
0811 18:45:16.680339 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:16.680427 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":233,"threads_started":1}
I20260811 18:45:17.145407 30999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:17.144549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.131:58433)
0811 18:45:17.145036 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:17.145118 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":279,"threads_started":1}
I20260811 18:45:17.207367 31000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:17.206640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.129:33703)
0811 18:45:17.207079 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:17.207159 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":235,"threads_started":1}
I20260811 18:45:17.682668 31004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:17.681847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.130:41303)
0811 18:45:17.682390 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:17.682475 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":293,"threads_started":1}
I20260811 18:45:18.147212 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:18.146439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.131:37903)
0811 18:45:18.146940 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:18.147031 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":279,"threads_started":1}
I20260811 18:45:18.209273 31009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:18.208428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.129:40967)
0811 18:45:18.208925 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:18.209027 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":262,"threads_started":1}
I20260811 18:45:18.684481 31010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:18.683661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.130:49901)
0811 18:45:18.684199 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:18.684302 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":270,"threads_started":1}
I20260811 18:45:19.148934 31011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:19.148186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.131:58191)
0811 18:45:19.148665 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:19.148760 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":243,"threads_started":1}
I20260811 18:45:19.211102 31012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:19.210288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.129:40673)
0811 18:45:19.210794 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:19.210890 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":220,"threads_started":1}
I20260811 18:45:19.686535 31015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:19.685645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.130:45069)
0811 18:45:19.686208 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:19.686313 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":252,"threads_started":1}
I20260811 18:45:20.150826 31017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:20.150010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.131:59281)
0811 18:45:20.150522 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:20.150600 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":212,"threads_started":1}
I20260811 18:45:20.212837 31019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:20.212061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.129:51223)
0811 18:45:20.212555 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:20.212650 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":253,"threads_started":1}
I20260811 18:45:20.688464 31020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:20.687617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.130:51455)
0811 18:45:20.688199 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:20.688286 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":331,"threads_started":1}
I20260811 18:45:21.152557 31022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:21.151829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.131:52817)
0811 18:45:21.152301 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:21.152389 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":240,"threads_started":1}
I20260811 18:45:21.214927 31024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:21.213857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.129:45069)
0811 18:45:21.214484 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:21.214683 (+ 199us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":296,"threads_started":1}
I20260811 18:45:21.690227 31025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:21.689460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.130:54545)
0811 18:45:21.689962 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:21.690048 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":249,"threads_started":1}
I20260811 18:45:22.154419 31027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:22.153633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.131:54619)
0811 18:45:22.154134 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:22.154238 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":255,"threads_started":1}
I20260811 18:45:22.216621 31028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:22.215836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.129:33983)
0811 18:45:22.216367 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:22.216452 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":291,"threads_started":1}
I20260811 18:45:22.692039 31033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:22.691285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.130:57881)
0811 18:45:22.691770 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:22.691870 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":250,"threads_started":1}
I20260811 18:45:23.156204 31034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:23.155405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.131:59059)
0811 18:45:23.155906 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:23.155994 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":292,"threads_started":1}
I20260811 18:45:23.218595 31037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:23.217758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.129:35039)
0811 18:45:23.218305 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:23.218408 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":270,"threads_started":1}
I20260811 18:45:23.693866 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:23.693030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.130:40837)
0811 18:45:23.693555 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:23.693673 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":292,"threads_started":1}
I20260811 18:45:24.158339 31040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:24.157448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.131:36443)
0811 18:45:24.157958 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:24.158050 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":265,"threads_started":1}
I20260811 18:45:24.220328 31043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:24.219598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.129:52735)
0811 18:45:24.220071 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:24.220162 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":252,"threads_started":1}
I20260811 18:45:24.695732 31045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:24.694964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.130:52373)
0811 18:45:24.695468 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:24.695563 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":275,"threads_started":1}
I20260811 18:45:25.160504 31046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:25.159645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.131:36423)
0811 18:45:25.160206 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:25.160318 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":297,"threads_started":1}
I20260811 18:45:25.222155 31047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:25.221379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.129:39439)
0811 18:45:25.221862 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:25.221962 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":226,"threads_started":1}
I20260811 18:45:25.697917 31048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:25.696909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.130:54605)
0811 18:45:25.697609 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:25.697715 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":354,"threads_started":1}
I20260811 18:45:26.162487 31049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:26.161587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.131:42397)
0811 18:45:26.162150 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:26.162294 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":319,"threads_started":1}
I20260811 18:45:26.223922 31050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:26.223112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.129:34403)
0811 18:45:26.223633 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:26.223729 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":277,"threads_started":1}
I20260811 18:45:26.699790 31052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:26.698993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.130:42405)
0811 18:45:26.699496 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:26.699589 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":245,"threads_started":1}
I20260811 18:45:27.164377 31053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:27.163542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.131:50833)
0811 18:45:27.164042 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:27.164133 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":250,"threads_started":1}
I20260811 18:45:27.225979 31057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:27.225168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.129:37231)
0811 18:45:27.225679 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:27.225779 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":258,"threads_started":1}
I20260811 18:45:27.702008 31058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:27.701023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.130:58407)
0811 18:45:27.701624 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:27.701748 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":350,"threads_started":1}
I20260811 18:45:28.140573 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 30176
I20260811 18:45:28.159799 31059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:28.142019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42236 (local address 127.30.185.129:38077)
0811 18:45:28.142858 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:28.142873 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:28.142913 (+ 40us) server_negotiation.cc:407] Connection header received
0811 18:45:28.144447 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:28.144457 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:28.144901 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:28.145274 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:28.146771 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:28.148060 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:28.149496 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:28.150273 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:28.150430 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:28.153289 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:28.153306 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:28.153324 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:28.153364 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:28.156453 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:28.157250 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:28.157257 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:28.157264 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:28.157459 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:28.157808 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:28.157812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:28.157814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:28.158424 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:28.158657 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:28.159141 (+ 484us) server_negotiation.cc:299] Negotiation successful
0811 18:45:28.159504 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":545,"threads_started":1}
I20260811 18:45:28.161311 30467 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:28.166522 31062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:28.165707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:36929 (local address 127.30.185.131:59247)
0811 18:45:28.166229 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:28.166332 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.190:36929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":272,"threads_started":1}
I20260811 18:45:28.249699 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 30176
I20260811 18:45:28.279125 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 30486
I20260811 18:45:28.297643 31064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:28.280105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45016 (local address 127.30.185.130:40149)
0811 18:45:28.280689 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:28.280704 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:28.280741 (+ 37us) server_negotiation.cc:407] Connection header received
0811 18:45:28.282402 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:28.282413 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:28.282846 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:28.283138 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:28.284594 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:28.286016 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:28.287294 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:28.287967 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:28.288102 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:28.290977 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:28.290985 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:28.291002 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:28.291042 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:28.294365 (+ 3323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:28.295044 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:28.295050 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:28.295057 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:28.295231 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:28.295602 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:28.295605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:28.295607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:28.296135 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:28.296392 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:28.296811 (+ 419us) server_negotiation.cc:299] Negotiation successful
0811 18:45:28.297323 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":296,"threads_started":1}
I20260811 18:45:28.299104 30721 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:28.382725 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 30486
I20260811 18:45:28.410939 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 30737
I20260811 18:45:28.430334 31066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:28.412011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46296 (local address 127.30.185.131:37527)
0811 18:45:28.412581 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:28.412596 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:28.412638 (+ 42us) server_negotiation.cc:407] Connection header received
0811 18:45:28.414503 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:28.414513 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:28.414965 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:28.415352 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:28.416875 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:28.418281 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:28.419713 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:28.420424 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:28.420635 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:28.423550 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:28.423558 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:28.423575 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:28.423614 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:28.426728 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:28.427459 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:28.427475 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:28.427483 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:28.427661 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:28.428077 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:28.428083 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:28.428086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:28.428643 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:28.429077 (+ 434us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:28.429555 (+ 478us) server_negotiation.cc:299] Negotiation successful
0811 18:45:28.430023 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":254,"threads_started":1}
I20260811 18:45:28.431771 30965 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:28.516495 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 30737
2026-08-11T18:45:28Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65662 ms)
[----------] 30 tests from SecurityITest (317804 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:45:28 dist-test-slave-z8x9 krb5kdc[31081](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:45:28 dist-test-slave-z8x9 krb5kdc[31081](info): set up 2 sockets
Aug 11 18:45:28 dist-test-slave-z8x9 krb5kdc[31081](info): commencing operation
krb5kdc: starting...
W20260811 18:45:29.948983 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.386s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:45:30 dist-test-slave-z8x9 krb5kdc[31081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:45:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:45:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:45:30.290457 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:44777
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:44777
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:45:30.639482 31102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:30.639891 31102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:30.640077 31102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:30.647624 31102 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:45:30.647835 31102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:30.647956 31102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:30.648069 31102 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:45:30.648144 31102 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:45:30.658767 31102 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:44777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:44777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:30.660567 31102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:30.662600 31102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:30.677431 31108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:30.677348 31114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:30.677407 31110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:30.678382 31102 server_base.cc:1034] running on GCE node
I20260811 18:45:30.679215 31102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:30.680629 31102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:30.681864 31102 hybrid_clock.cc:648] HybridClock initialized: now 1786473930681815 us; error 38 us; skew 500 ppm
I20260811 18:45:30.682271 31102 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:45:30 dist-test-slave-z8x9 krb5kdc[31081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473930, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:45:30.685653 31102 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:45:30.687309 31102 webserver.cc:460] Webserver started at http://127.30.185.190:41081/ using document root <none> and password file <none>
I20260811 18:45:30.688030 31102 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:30.688184 31102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:30.688516 31102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:30.691809 31102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "846be1e75b9243fc8ca7a9e14b23e082"
format_stamp: "Formatted at 2026-08-11 18:45:30 on dist-test-slave-z8x9"
server_key: "0c2c31e4c52aeacd2f181c4f816e45f3"
server_key_iv: "4954dc624077bb03d83ece27f566ceed"
server_key_version: "encryptionkey@0"
I20260811 18:45:30.692617 31102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "846be1e75b9243fc8ca7a9e14b23e082"
format_stamp: "Formatted at 2026-08-11 18:45:30 on dist-test-slave-z8x9"
server_key: "0c2c31e4c52aeacd2f181c4f816e45f3"
server_key_iv: "4954dc624077bb03d83ece27f566ceed"
server_key_version: "encryptionkey@0"
I20260811 18:45:30.698028 31102 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 18:45:30.701623 31131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:30.702971 31102 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:45:30.703173 31102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "846be1e75b9243fc8ca7a9e14b23e082"
format_stamp: "Formatted at 2026-08-11 18:45:30 on dist-test-slave-z8x9"
server_key: "0c2c31e4c52aeacd2f181c4f816e45f3"
server_key_iv: "4954dc624077bb03d83ece27f566ceed"
server_key_version: "encryptionkey@0"
I20260811 18:45:30.703399 31102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:30.726725 31102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:30.731464 31102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:30.731781 31102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:30.751698 31102 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:44777
I20260811 18:45:30.751722 31207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:44777 every 8 connection(s)
I20260811 18:45:30.753412 31102 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:45:30.757565 31209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:30.762979 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 31102
I20260811 18:45:30.763236 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:45:30.764062 31462 external_mini_cluster.cc:1428] Setting key 26061bceef00c0e705323665ab446fd9
I20260811 18:45:30.769709 31209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082: Bootstrap starting.
I20260811 18:45:30.773603 31209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:30.774884 31209 log.cc:824] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:30.778578 31209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082: No bootstrap required, opened a new log
Aug 11 18:45:30 dist-test-slave-z8x9 krb5kdc[31081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:45:30.787051 31209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "846be1e75b9243fc8ca7a9e14b23e082" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44777 } }
I20260811 18:45:30.787457 31209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:30.787597 31209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 846be1e75b9243fc8ca7a9e14b23e082, State: Initialized, Role: FOLLOWER
I20260811 18:45:30.788270 31209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "846be1e75b9243fc8ca7a9e14b23e082" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44777 } }
I20260811 18:45:30.788548 31209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:45:30.788725 31209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:45:30.788928 31209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:30.789404 31214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:30.766345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47906 (local address 127.30.185.190:44777)
0811 18:45:30.767250 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:30.767262 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:30.767305 (+ 43us) server_negotiation.cc:407] Connection header received
0811 18:45:30.769046 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:30.769069 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:30.769525 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:30.769860 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:30.771831 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:30.773130 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:30.775118 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:30.776246 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:30.776428 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:30.781565 (+ 5137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:30.781573 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:30.781587 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:30.781631 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:30.784857 (+ 3226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:30.785653 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:30.785661 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:30.785667 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:30.785861 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:30.786285 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:30.786289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:30.786291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:30.786828 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:30.787134 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:30.787643 (+ 509us) server_negotiation.cc:299] Negotiation successful
0811 18:45:30.788119 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":431,"threads_started":1}
I20260811 18:45:30.791862 31209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "846be1e75b9243fc8ca7a9e14b23e082" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44777 } }
I20260811 18:45:30.792438 31209 leader_election.cc:302] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 846be1e75b9243fc8ca7a9e14b23e082; no voters:
I20260811 18:45:30.793082 31209 leader_election.cc:288] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:45:30.793247 31216 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:45:30.794646 31216 raft_consensus.cc:686] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082 [term 1 LEADER]: Becoming Leader. State: Replica: 846be1e75b9243fc8ca7a9e14b23e082, State: Running, Role: LEADER
I20260811 18:45:30.795181 31216 consensus_queue.cc:237] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "846be1e75b9243fc8ca7a9e14b23e082" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44777 } }
I20260811 18:45:30.795526 31209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:45:30.798027 31217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "846be1e75b9243fc8ca7a9e14b23e082" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "846be1e75b9243fc8ca7a9e14b23e082" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44777 } } }
I20260811 18:45:30.797999 31219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 846be1e75b9243fc8ca7a9e14b23e082. Latest consensus state: current_term: 1 leader_uuid: "846be1e75b9243fc8ca7a9e14b23e082" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "846be1e75b9243fc8ca7a9e14b23e082" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44777 } } }
I20260811 18:45:30.798542 31219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082 [sys.catalog]: This master's current role is: LEADER
I20260811 18:45:30.798554 31217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082 [sys.catalog]: This master's current role is: LEADER
I20260811 18:45:30.800256 31229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:45:30.805567 31229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:45:30.814636 31229 catalog_manager.cc:1347] Generated new cluster ID: a7a0a0f71d7843b5870df8d0e0b64635
I20260811 18:45:30.814782 31229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:45:30.847393 31229 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:45:30.849427 31229 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:45:30.859797 31229 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082: Generated new TSK 0
I20260811 18:45:30.860886 31229 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.185.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
I20260811 18:45:30.953473 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:44777
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:45:31.310947 31255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:31.311232 31255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:31.311435 31255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:31.318832 31255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:31.318945 31255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:31.319128 31255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:45:31.329105 31255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:44777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:31.330462 31255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:31.331823 31255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:31.345806 31265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:31.345860 31263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:31.345903 31262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:31.352181 31255 server_base.cc:1034] running on GCE node
I20260811 18:45:31.352948 31255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:31.354094 31255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:31.355319 31255 hybrid_clock.cc:648] HybridClock initialized: now 1786473931355264 us; error 42 us; skew 500 ppm
I20260811 18:45:31.355661 31255 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:45:31 dist-test-slave-z8x9 krb5kdc[31081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473931, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:45:31.362103 31255 init.cc:377] Logged in from keytab as kudu/127.30.185.129@KRBTEST.COM (short username kudu)
I20260811 18:45:31.364027 31255 webserver.cc:460] Webserver started at http://127.30.185.129:33257/ using document root <none> and password file <none>
I20260811 18:45:31.364809 31255 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:31.364969 31255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:31.365376 31255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:31.368432 31255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "2c551fd550dc448cb842112faaff1d6d"
format_stamp: "Formatted at 2026-08-11 18:45:31 on dist-test-slave-z8x9"
server_key: "fdd718bab0644a5431d1f73b922a0f81"
server_key_iv: "bd1a2d35cf72a2bbaef141294966bbfb"
server_key_version: "encryptionkey@0"
I20260811 18:45:31.369302 31255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "2c551fd550dc448cb842112faaff1d6d"
format_stamp: "Formatted at 2026-08-11 18:45:31 on dist-test-slave-z8x9"
server_key: "fdd718bab0644a5431d1f73b922a0f81"
server_key_iv: "bd1a2d35cf72a2bbaef141294966bbfb"
server_key_version: "encryptionkey@0"
I20260811 18:45:31.374462 31255 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 18:45:31.378005 31276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:31.379400 31255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:45:31.379568 31255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "2c551fd550dc448cb842112faaff1d6d"
format_stamp: "Formatted at 2026-08-11 18:45:31 on dist-test-slave-z8x9"
server_key: "fdd718bab0644a5431d1f73b922a0f81"
server_key_iv: "bd1a2d35cf72a2bbaef141294966bbfb"
server_key_version: "encryptionkey@0"
I20260811 18:45:31.379787 31255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:31.396872 31255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:31.401491 31255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:31.401808 31255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:31.402933 31255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:45:31.404520 31255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:45:31.404631 31255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:31.404785 31255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:45:31.404865 31255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:31.439811 31255 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:39083
I20260811 18:45:31.439868 31474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:39083 every 8 connection(s)
I20260811 18:45:31.441228 31255 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:45:31.449643 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 31255
I20260811 18:45:31.449939 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:45:31.450608 31462 external_mini_cluster.cc:1428] Setting key d7fd32909a4e607e1bfbdd11b80025ab
Aug 11 18:45:31 dist-test-slave-z8x9 krb5kdc[31081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473931, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:45:31.463900 31481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:31.443851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:43781 (local address 127.30.185.190:44777)
0811 18:45:31.444281 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:31.444285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:31.445337 (+ 1052us) server_negotiation.cc:407] Connection header received
0811 18:45:31.447209 (+ 1872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:31.447215 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:31.447312 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:31.447437 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:31.448948 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:31.449728 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:31.451571 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:31.452177 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:31.452311 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:31.456584 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:31.456594 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:31.456602 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:31.456650 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:31.460282 (+ 3632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:31.461061 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:31.461069 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:31.461075 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:31.461260 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:31.461748 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:31.461756 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:31.461762 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:31.462153 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:31.462315 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:31.463167 (+ 852us) server_negotiation.cc:299] Negotiation successful
0811 18:45:31.463439 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":210,"threads_started":1}
I20260811 18:45:31.464875 31484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:31.444428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:44777 (local address 127.30.185.129:43781)
0811 18:45:31.445118 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:31.445156 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:31.446932 (+ 1776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:31.447679 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:31.447690 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:31.448216 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:45:31.448770 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:31.448787 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:31.449918 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:31.449923 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:31.451409 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:31.451426 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:31.452445 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:31.452453 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:31.452730 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:31.453546 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:45:31.453578 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:45:31.456336 (+ 2758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:45:31.460460 (+ 4124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:45:31.460467 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:45:31.460481 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:45:31.460938 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:45:31.461406 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:45:31.461410 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:45:31.461413 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:45:31.461634 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:45:31.462294 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:45:31.462304 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:45:31.462742 (+ 438us) client_negotiation.cc:769] Sending connection context
0811 18:45:31.463024 (+ 282us) client_negotiation.cc:240] Negotiation successful
0811 18:45:31.463702 (+ 678us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":280,"threads_started":1}
I20260811 18:45:31.466492 31475 heartbeater.cc:347] Connected to a master server at 127.30.185.190:44777
I20260811 18:45:31.466796 31475 heartbeater.cc:464] Registering TS with master...
I20260811 18:45:31.467610 31475 heartbeater.cc:511] Master 127.30.185.190:44777 requested a full tablet report, sending...
I20260811 18:45:31.469704 31155 ts_manager.cc:194] Registered new tserver with Master: 2c551fd550dc448cb842112faaff1d6d (127.30.185.129:39083)
I20260811 18:45:31.471124 31155 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:43781
WARNING: no policy specified for kudu/127.30.185.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
I20260811 18:45:31.525105 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:44777
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:45:31.881739 31490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:31.882030 31490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:31.882220 31490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:31.889609 31490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:31.889736 31490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:31.889921 31490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:45:31.899796 31490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:44777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:31.901229 31490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:31.902652 31490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:31.917606 31495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:31.917654 31497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:31.917690 31503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:31.918526 31490 server_base.cc:1034] running on GCE node
I20260811 18:45:31.919231 31490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:31.920197 31490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:31.921442 31490 hybrid_clock.cc:648] HybridClock initialized: now 1786473931921387 us; error 44 us; skew 500 ppm
I20260811 18:45:31.921795 31490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:45:31 dist-test-slave-z8x9 krb5kdc[31081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473931, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:45:31.925117 31490 init.cc:377] Logged in from keytab as kudu/127.30.185.130@KRBTEST.COM (short username kudu)
I20260811 18:45:31.926751 31490 webserver.cc:460] Webserver started at http://127.30.185.130:40181/ using document root <none> and password file <none>
I20260811 18:45:31.927451 31490 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:31.927613 31490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:31.927948 31490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:31.931049 31490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "1defdb5c159d4a898ac0050344109e60"
format_stamp: "Formatted at 2026-08-11 18:45:31 on dist-test-slave-z8x9"
server_key: "61ddb5834fcb381a5ed27197abaa6b6f"
server_key_iv: "cbb2326a81ae081a683a9c3dc0a6f165"
server_key_version: "encryptionkey@0"
I20260811 18:45:31.931797 31490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "1defdb5c159d4a898ac0050344109e60"
format_stamp: "Formatted at 2026-08-11 18:45:31 on dist-test-slave-z8x9"
server_key: "61ddb5834fcb381a5ed27197abaa6b6f"
server_key_iv: "cbb2326a81ae081a683a9c3dc0a6f165"
server_key_version: "encryptionkey@0"
I20260811 18:45:31.937404 31490 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 18:45:31.942078 31525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:31.943605 31490 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:45:31.943809 31490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "1defdb5c159d4a898ac0050344109e60"
format_stamp: "Formatted at 2026-08-11 18:45:31 on dist-test-slave-z8x9"
server_key: "61ddb5834fcb381a5ed27197abaa6b6f"
server_key_iv: "cbb2326a81ae081a683a9c3dc0a6f165"
server_key_version: "encryptionkey@0"
I20260811 18:45:31.944048 31490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:31.977464 31490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:31.982213 31490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:31.982529 31490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:31.983690 31490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:45:31.985333 31490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:45:31.985476 31490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:31.985628 31490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:45:31.985713 31490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:32.021481 31490 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:44119
I20260811 18:45:32.021528 31710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:44119 every 8 connection(s)
I20260811 18:45:32.022841 31490 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:45:32.031253 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 31490
I20260811 18:45:32.031476 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:45:32.031944 31462 external_mini_cluster.cc:1428] Setting key 4bf79fa965e1123074f85bbd81804145
Aug 11 18:45:32 dist-test-slave-z8x9 krb5kdc[31081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473931, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:45:32.044250 31715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:32.025497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:52857 (local address 127.30.185.190:44777)
0811 18:45:32.025954 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:32.025958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:32.026993 (+ 1035us) server_negotiation.cc:407] Connection header received
0811 18:45:32.028998 (+ 2005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:32.029004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:32.029108 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:32.029279 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:32.030906 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.031754 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.033629 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.034243 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.034392 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:32.038011 (+ 3619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:32.038017 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:32.038020 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:32.038051 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:32.040683 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:32.041539 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:32.041548 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:32.041551 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:32.041680 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:32.042171 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:32.042178 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:32.042184 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:32.042602 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:32.042758 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:32.043612 (+ 854us) server_negotiation.cc:299] Negotiation successful
0811 18:45:32.043850 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":231,"threads_started":1}
I20260811 18:45:32.045372 31716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:32.026103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:44777 (local address 127.30.185.130:52857)
0811 18:45:32.026821 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:32.026864 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:32.028700 (+ 1836us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:32.029549 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:32.029558 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:32.030103 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:45:32.030712 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:32.030729 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.031927 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.031931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:32.033477 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:32.033490 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.034493 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.034501 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:32.034734 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:32.035482 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:45:32.035505 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:45:32.037781 (+ 2276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:45:32.040830 (+ 3049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:45:32.040840 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:45:32.040857 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:45:32.041399 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:45:32.041795 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:45:32.041800 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:45:32.041804 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:45:32.042058 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:45:32.042850 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:45:32.042858 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:45:32.043238 (+ 380us) client_negotiation.cc:769] Sending connection context
0811 18:45:32.043519 (+ 281us) client_negotiation.cc:240] Negotiation successful
0811 18:45:32.044074 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":297,"threads_started":1}
I20260811 18:45:32.046991 31713 heartbeater.cc:347] Connected to a master server at 127.30.185.190:44777
I20260811 18:45:32.047307 31713 heartbeater.cc:464] Registering TS with master...
I20260811 18:45:32.048163 31713 heartbeater.cc:511] Master 127.30.185.190:44777 requested a full tablet report, sending...
I20260811 18:45:32.049809 31155 ts_manager.cc:194] Registered new tserver with Master: 1defdb5c159d4a898ac0050344109e60 (127.30.185.130:44119)
I20260811 18:45:32.050683 31155 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:52857
WARNING: no policy specified for kudu/127.30.185.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
I20260811 18:45:32.105041 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:44777
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:45:32.454969 31721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:32.455296 31721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:32.455470 31721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:32.463073 31721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:32.463207 31721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:32.463411 31721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:45:32.473906 31721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:44777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:32.474777 31475 heartbeater.cc:503] Master 127.30.185.190:44777 was elected leader, sending a full tablet report...
I20260811 18:45:32.475570 31721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:32.477308 31721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:32.491312 31729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:32.491456 31735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:32.491900 31727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:32.492497 31721 server_base.cc:1034] running on GCE node
I20260811 18:45:32.493232 31721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:32.494221 31721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:32.495473 31721 hybrid_clock.cc:648] HybridClock initialized: now 1786473932495406 us; error 42 us; skew 500 ppm
I20260811 18:45:32.495814 31721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:45:32 dist-test-slave-z8x9 krb5kdc[31081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473932, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:45:32.499382 31721 init.cc:377] Logged in from keytab as kudu/127.30.185.131@KRBTEST.COM (short username kudu)
I20260811 18:45:32.500967 31721 webserver.cc:460] Webserver started at http://127.30.185.131:43185/ using document root <none> and password file <none>
I20260811 18:45:32.501896 31721 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:32.502067 31721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:32.502424 31721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:32.505699 31721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "b39aadc0af474fa2a8de3b6923da4526"
format_stamp: "Formatted at 2026-08-11 18:45:32 on dist-test-slave-z8x9"
server_key: "b150db425975d8e098ccecba16ef643d"
server_key_iv: "69e6c3ddea7ea8c6f1360013837f1b05"
server_key_version: "encryptionkey@0"
I20260811 18:45:32.506626 31721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "b39aadc0af474fa2a8de3b6923da4526"
format_stamp: "Formatted at 2026-08-11 18:45:32 on dist-test-slave-z8x9"
server_key: "b150db425975d8e098ccecba16ef643d"
server_key_iv: "69e6c3ddea7ea8c6f1360013837f1b05"
server_key_version: "encryptionkey@0"
I20260811 18:45:32.512569 31721 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 18:45:32.516415 31745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:32.517927 31721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:45:32.518070 31721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "b39aadc0af474fa2a8de3b6923da4526"
format_stamp: "Formatted at 2026-08-11 18:45:32 on dist-test-slave-z8x9"
server_key: "b150db425975d8e098ccecba16ef643d"
server_key_iv: "69e6c3ddea7ea8c6f1360013837f1b05"
server_key_version: "encryptionkey@0"
I20260811 18:45:32.518247 31721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:32.532426 31721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:32.536585 31721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:32.536832 31721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:32.537875 31721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:45:32.539166 31721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:45:32.539242 31721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:32.539326 31721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:45:32.539382 31721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:32.574015 31721 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:44369
I20260811 18:45:32.574067 31952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:44369 every 8 connection(s)
I20260811 18:45:32.575323 31721 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:45:32.578341 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 31721
I20260811 18:45:32.578557 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:45:32.579046 31462 external_mini_cluster.cc:1428] Setting key 9b7af168735ff2cab2e6c6903cc54e17
Aug 11 18:45:32 dist-test-slave-z8x9 krb5kdc[31081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473932, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:45:32.596477 31955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:32.577926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:35253 (local address 127.30.185.190:44777)
0811 18:45:32.578486 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:32.578493 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:32.579818 (+ 1325us) server_negotiation.cc:407] Connection header received
0811 18:45:32.581514 (+ 1696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:32.581520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:32.581613 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:32.581733 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:32.583312 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.584099 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.585965 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.586343 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.586439 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:32.589865 (+ 3426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:32.589870 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:32.589873 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:32.589905 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:32.592575 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:32.593549 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:32.593554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:32.593557 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:32.593719 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:32.594546 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:32.594554 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:32.594560 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:32.594985 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:32.595113 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:32.595943 (+ 830us) server_negotiation.cc:299] Negotiation successful
0811 18:45:32.596123 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":250,"threads_started":1}
I20260811 18:45:32.597450 31956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:32.578534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:44777 (local address 127.30.185.131:35253)
0811 18:45:32.579424 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:32.579471 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:32.581222 (+ 1751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:32.582008 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:32.582016 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:32.582521 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:45:32.583078 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:32.583095 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.584234 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.584238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:32.585812 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:32.585825 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.586555 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.586560 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:32.586758 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:32.587392 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:45:32.587413 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:45:32.589667 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:45:32.592718 (+ 3051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:45:32.592726 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:45:32.592743 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:45:32.593411 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:45:32.593833 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:45:32.593841 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:45:32.593844 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:45:32.594318 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:45:32.595111 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:45:32.595117 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:45:32.595557 (+ 440us) client_negotiation.cc:769] Sending connection context
0811 18:45:32.595801 (+ 244us) client_negotiation.cc:240] Negotiation successful
0811 18:45:32.596209 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":347,"threads_started":1}
I20260811 18:45:32.598839 31953 heartbeater.cc:347] Connected to a master server at 127.30.185.190:44777
I20260811 18:45:32.599123 31953 heartbeater.cc:464] Registering TS with master...
I20260811 18:45:32.599866 31953 heartbeater.cc:511] Master 127.30.185.190:44777 requested a full tablet report, sending...
I20260811 18:45:32.601368 31155 ts_manager.cc:194] Registered new tserver with Master: b39aadc0af474fa2a8de3b6923da4526 (127.30.185.131:44369)
I20260811 18:45:32.602295 31155 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:35253
I20260811 18:45:32.607394 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:45:32.625502 31955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:32.612184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54250 (local address 127.30.185.190:44777)
0811 18:45:32.612399 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:32.612404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:32.612823 (+ 419us) server_negotiation.cc:407] Connection header received
0811 18:45:32.612975 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:32.612979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:32.613062 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:32.613246 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:32.614653 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.615393 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.616733 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.617158 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.617326 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:32.619824 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:32.619830 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:32.619834 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:32.619867 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:32.623028 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:32.623674 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:32.623679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:32.623681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:32.623818 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:32.624155 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:32.624159 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:32.624161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:32.624504 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:32.624701 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:32.625017 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 18:45:32.625266 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260811 18:45:32.629855 31155 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:45:32.636637 31155 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:45:32.663672 31970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:32.649409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:44369 (local address 127.0.0.1:51462)
0811 18:45:32.650173 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:32.650230 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:32.650432 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:32.652830 (+ 2398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:32.652841 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:32.652885 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:32.653511 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:32.653539 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.656080 (+ 2541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.656090 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:32.659667 (+ 3577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:32.659691 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.662342 (+ 2651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.662350 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:32.662636 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:32.662719 (+ 83us) client_negotiation.cc:769] Sending connection context
0811 18:45:32.662859 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 18:45:32.663121 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":302,"threads_started":1}
I20260811 18:45:32.666224 31972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:32.649730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51462 (local address 127.30.185.131:44369)
0811 18:45:32.652137 (+ 2407us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:32.652144 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:32.652178 (+ 34us) server_negotiation.cc:407] Connection header received
0811 18:45:32.652297 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:32.652306 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:32.652556 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:32.653387 (+ 831us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:32.653728 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.655872 (+ 2144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.659955 (+ 4083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.662058 (+ 2103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.664837 (+ 2779us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:32.665100 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:32.665211 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 18:45:32.665754 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2259,"thread_start_us":2096,"threads_started":1}
I20260811 18:45:32.666909 31971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:32.649613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36166 (local address 127.30.185.130:44119)
0811 18:45:32.651021 (+ 1408us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:32.651034 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:32.651912 (+ 878us) server_negotiation.cc:407] Connection header received
0811 18:45:32.652295 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:32.652308 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:32.652593 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:32.652794 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:32.655794 (+ 3000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.659498 (+ 3704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.663845 (+ 4347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.665383 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.665544 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:32.665730 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:32.666367 (+ 637us) server_negotiation.cc:299] Negotiation successful
0811 18:45:32.666570 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1266,"thread_start_us":1150,"threads_started":1}
I20260811 18:45:32.667814 31974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:32.650177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:44119 (local address 127.0.0.1:36166)
0811 18:45:32.651807 (+ 1630us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:32.651824 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:32.652030 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:32.653149 (+ 1119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:32.653157 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:32.653246 (+ 89us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:32.655527 (+ 2281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:32.655545 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.659840 (+ 4295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.659845 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:32.663617 (+ 3772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:32.663635 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.665931 (+ 2296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.665939 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:32.666116 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:32.666165 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 18:45:32.667110 (+ 945us) client_negotiation.cc:240] Negotiation successful
0811 18:45:32.667340 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1491,"thread_start_us":1321,"threads_started":1}
I20260811 18:45:32.670555 31843 tablet_service.cc:1467] Processing CreateTablet for tablet 55e25afc90d049d384ce6cc4dcedc2e1 (DEFAULT_TABLE table=test-table [id=2e1739cf47424fb9b6bdfa5126880f83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:45:32.672246 31843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55e25afc90d049d384ce6cc4dcedc2e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:32.672346 31606 tablet_service.cc:1467] Processing CreateTablet for tablet 55e25afc90d049d384ce6cc4dcedc2e1 (DEFAULT_TABLE table=test-table [id=2e1739cf47424fb9b6bdfa5126880f83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:45:32.673867 31606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55e25afc90d049d384ce6cc4dcedc2e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:32.677558 31970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:32.651891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:39083 (local address 127.0.0.1:41736)
0811 18:45:32.664385 (+ 12494us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:32.664410 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:32.664540 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:32.671538 (+ 6998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:32.671543 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:32.671576 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:32.671767 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:32.671783 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.673537 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.673549 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:32.675312 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:32.675333 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.676712 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.676721 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:32.676910 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:32.676942 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 18:45:32.677070 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 18:45:32.677255 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12346,"thread_start_us":19380,"threads_started":1}
I20260811 18:45:32.677646 31978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:32.652511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41736 (local address 127.30.185.129:39083)
0811 18:45:32.670319 (+ 17808us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:32.670331 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:32.670369 (+ 38us) server_negotiation.cc:407] Connection header received
0811 18:45:32.670506 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:32.670519 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:32.670815 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:32.671940 (+ 1125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:32.672154 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.673385 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.675536 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.676530 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.676759 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:32.676939 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:32.677117 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 18:45:32.677315 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17566,"thread_start_us":18999,"threads_started":1}
I20260811 18:45:32.680956 31354 tablet_service.cc:1467] Processing CreateTablet for tablet 55e25afc90d049d384ce6cc4dcedc2e1 (DEFAULT_TABLE table=test-table [id=2e1739cf47424fb9b6bdfa5126880f83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:45:32.682548 31354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55e25afc90d049d384ce6cc4dcedc2e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:32.687817 31979 tablet_bootstrap.cc:492] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526: Bootstrap starting.
I20260811 18:45:32.691254 31981 tablet_bootstrap.cc:492] T 55e25afc90d049d384ce6cc4dcedc2e1 P 1defdb5c159d4a898ac0050344109e60: Bootstrap starting.
I20260811 18:45:32.692236 31979 tablet_bootstrap.cc:654] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:32.693670 31979 log.cc:824] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:32.694226 31981 tablet_bootstrap.cc:654] T 55e25afc90d049d384ce6cc4dcedc2e1 P 1defdb5c159d4a898ac0050344109e60: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:32.695497 31981 log.cc:824] T 55e25afc90d049d384ce6cc4dcedc2e1 P 1defdb5c159d4a898ac0050344109e60: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:32.696988 31985 tablet_bootstrap.cc:492] T 55e25afc90d049d384ce6cc4dcedc2e1 P 2c551fd550dc448cb842112faaff1d6d: Bootstrap starting.
I20260811 18:45:32.697384 31979 tablet_bootstrap.cc:492] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526: No bootstrap required, opened a new log
I20260811 18:45:32.697685 31979 ts_tablet_manager.cc:1397] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.009s
I20260811 18:45:32.698971 31981 tablet_bootstrap.cc:492] T 55e25afc90d049d384ce6cc4dcedc2e1 P 1defdb5c159d4a898ac0050344109e60: No bootstrap required, opened a new log
I20260811 18:45:32.699234 31981 ts_tablet_manager.cc:1397] T 55e25afc90d049d384ce6cc4dcedc2e1 P 1defdb5c159d4a898ac0050344109e60: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.001s
I20260811 18:45:32.700754 31985 tablet_bootstrap.cc:654] T 55e25afc90d049d384ce6cc4dcedc2e1 P 2c551fd550dc448cb842112faaff1d6d: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:32.702580 31985 log.cc:824] T 55e25afc90d049d384ce6cc4dcedc2e1 P 2c551fd550dc448cb842112faaff1d6d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:32.706979 31985 tablet_bootstrap.cc:492] T 55e25afc90d049d384ce6cc4dcedc2e1 P 2c551fd550dc448cb842112faaff1d6d: No bootstrap required, opened a new log
I20260811 18:45:32.706764 31979 raft_consensus.cc:348] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defdb5c159d4a898ac0050344109e60" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44119 } } peers { permanent_uuid: "b39aadc0af474fa2a8de3b6923da4526" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 44369 } } peers { permanent_uuid: "2c551fd550dc448cb842112faaff1d6d" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 39083 } }
I20260811 18:45:32.707262 31979 raft_consensus.cc:374] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:32.707265 31985 ts_tablet_manager.cc:1397] T 55e25afc90d049d384ce6cc4dcedc2e1 P 2c551fd550dc448cb842112faaff1d6d: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260811 18:45:32.707396 31979 raft_consensus.cc:729] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b39aadc0af474fa2a8de3b6923da4526, State: Initialized, Role: FOLLOWER
I20260811 18:45:32.708057 31979 consensus_queue.cc:260] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defdb5c159d4a898ac0050344109e60" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44119 } } peers { permanent_uuid: "b39aadc0af474fa2a8de3b6923da4526" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 44369 } } peers { permanent_uuid: "2c551fd550dc448cb842112faaff1d6d" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 39083 } }
I20260811 18:45:32.709267 31953 heartbeater.cc:503] Master 127.30.185.190:44777 was elected leader, sending a full tablet report...
I20260811 18:45:32.709931 31979 ts_tablet_manager.cc:1428] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260811 18:45:32.710831 31981 raft_consensus.cc:348] T 55e25afc90d049d384ce6cc4dcedc2e1 P 1defdb5c159d4a898ac0050344109e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defdb5c159d4a898ac0050344109e60" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44119 } } peers { permanent_uuid: "b39aadc0af474fa2a8de3b6923da4526" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 44369 } } peers { permanent_uuid: "2c551fd550dc448cb842112faaff1d6d" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 39083 } }
I20260811 18:45:32.711292 31981 raft_consensus.cc:374] T 55e25afc90d049d384ce6cc4dcedc2e1 P 1defdb5c159d4a898ac0050344109e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:32.711397 31981 raft_consensus.cc:729] T 55e25afc90d049d384ce6cc4dcedc2e1 P 1defdb5c159d4a898ac0050344109e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1defdb5c159d4a898ac0050344109e60, State: Initialized, Role: FOLLOWER
I20260811 18:45:32.712096 31981 consensus_queue.cc:260] T 55e25afc90d049d384ce6cc4dcedc2e1 P 1defdb5c159d4a898ac0050344109e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defdb5c159d4a898ac0050344109e60" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44119 } } peers { permanent_uuid: "b39aadc0af474fa2a8de3b6923da4526" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 44369 } } peers { permanent_uuid: "2c551fd550dc448cb842112faaff1d6d" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 39083 } }
I20260811 18:45:32.713301 31713 heartbeater.cc:503] Master 127.30.185.190:44777 was elected leader, sending a full tablet report...
I20260811 18:45:32.715399 31981 ts_tablet_manager.cc:1428] T 55e25afc90d049d384ce6cc4dcedc2e1 P 1defdb5c159d4a898ac0050344109e60: Time spent starting tablet: real 0.016s user 0.011s sys 0.003s
I20260811 18:45:32.718323 31985 raft_consensus.cc:348] T 55e25afc90d049d384ce6cc4dcedc2e1 P 2c551fd550dc448cb842112faaff1d6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defdb5c159d4a898ac0050344109e60" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44119 } } peers { permanent_uuid: "b39aadc0af474fa2a8de3b6923da4526" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 44369 } } peers { permanent_uuid: "2c551fd550dc448cb842112faaff1d6d" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 39083 } }
I20260811 18:45:32.718660 31985 raft_consensus.cc:374] T 55e25afc90d049d384ce6cc4dcedc2e1 P 2c551fd550dc448cb842112faaff1d6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:32.718744 31985 raft_consensus.cc:729] T 55e25afc90d049d384ce6cc4dcedc2e1 P 2c551fd550dc448cb842112faaff1d6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c551fd550dc448cb842112faaff1d6d, State: Initialized, Role: FOLLOWER
I20260811 18:45:32.719215 31985 consensus_queue.cc:260] T 55e25afc90d049d384ce6cc4dcedc2e1 P 2c551fd550dc448cb842112faaff1d6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defdb5c159d4a898ac0050344109e60" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44119 } } peers { permanent_uuid: "b39aadc0af474fa2a8de3b6923da4526" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 44369 } } peers { permanent_uuid: "2c551fd550dc448cb842112faaff1d6d" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 39083 } }
I20260811 18:45:32.720803 31985 ts_tablet_manager.cc:1428] T 55e25afc90d049d384ce6cc4dcedc2e1 P 2c551fd550dc448cb842112faaff1d6d: Time spent starting tablet: real 0.013s user 0.011s sys 0.004s
W20260811 18:45:32.775014 31714 tablet.cc:2369] T 55e25afc90d049d384ce6cc4dcedc2e1 P 1defdb5c159d4a898ac0050344109e60: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:45:32.827042 31954 tablet.cc:2369] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:45:32.855994 31988 raft_consensus.cc:482] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:45:32.856348 31988 raft_consensus.cc:504] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defdb5c159d4a898ac0050344109e60" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44119 } } peers { permanent_uuid: "b39aadc0af474fa2a8de3b6923da4526" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 44369 } } peers { permanent_uuid: "2c551fd550dc448cb842112faaff1d6d" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 39083 } }
I20260811 18:45:32.858325 31988 leader_election.cc:288] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1defdb5c159d4a898ac0050344109e60 (127.30.185.130:44119), 2c551fd550dc448cb842112faaff1d6d (127.30.185.129:39083)
I20260811 18:45:32.865967 31978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:32.858810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:48791 (local address 127.30.185.129:39083)
0811 18:45:32.859039 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:32.859044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:32.859283 (+ 239us) server_negotiation.cc:407] Connection header received
0811 18:45:32.859509 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:32.859514 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:32.859602 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:32.859701 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:32.860180 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.861225 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.863775 (+ 2550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.864983 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.865152 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:32.865270 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:32.865598 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 18:45:32.865701 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 18:45:32.866016 31956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:32.858803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:39083 (local address 127.30.185.131:48791)
0811 18:45:32.859141 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:32.859157 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:32.859332 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:32.859760 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:32.859767 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:32.859810 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:32.860029 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:32.860046 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.861359 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.861363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:32.863618 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:32.863634 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.865143 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.865147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:32.865341 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:32.865398 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 18:45:32.865531 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 18:45:32.865678 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":116}
I20260811 18:45:32.866226 31971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:32.858782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:54239 (local address 127.30.185.130:44119)
0811 18:45:32.859011 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:32.859020 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:32.859873 (+ 853us) server_negotiation.cc:407] Connection header received
0811 18:45:32.860017 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:32.860021 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:32.860079 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:32.860196 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:32.860679 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.861641 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.864021 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.865288 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.865446 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:32.865553 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:32.865844 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 18:45:32.865978 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 18:45:32.866597 31992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:32.858921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:44119 (local address 127.30.185.131:54239)
0811 18:45:32.859751 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:32.859768 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:32.859898 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:32.860219 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:32.860222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:32.860241 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:32.860529 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:32.860536 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.861773 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.861779 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:32.863881 (+ 2102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:32.863891 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.865495 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.865504 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:32.865677 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:32.865705 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 18:45:32.865798 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 18:45:32.865922 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"mutex_wait_us":21,"thread_start_us":473,"threads_started":1}
I20260811 18:45:32.867461 31395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55e25afc90d049d384ce6cc4dcedc2e1" candidate_uuid: "b39aadc0af474fa2a8de3b6923da4526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c551fd550dc448cb842112faaff1d6d" is_pre_election: true
I20260811 18:45:32.867698 31637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55e25afc90d049d384ce6cc4dcedc2e1" candidate_uuid: "b39aadc0af474fa2a8de3b6923da4526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1defdb5c159d4a898ac0050344109e60" is_pre_election: true
I20260811 18:45:32.868013 31395 raft_consensus.cc:2454] T 55e25afc90d049d384ce6cc4dcedc2e1 P 2c551fd550dc448cb842112faaff1d6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b39aadc0af474fa2a8de3b6923da4526 in term 0.
I20260811 18:45:32.868247 31637 raft_consensus.cc:2454] T 55e25afc90d049d384ce6cc4dcedc2e1 P 1defdb5c159d4a898ac0050344109e60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b39aadc0af474fa2a8de3b6923da4526 in term 0.
I20260811 18:45:32.868834 31746 leader_election.cc:302] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c551fd550dc448cb842112faaff1d6d, b39aadc0af474fa2a8de3b6923da4526; no voters:
I20260811 18:45:32.869268 31988 raft_consensus.cc:2781] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:45:32.869422 31988 raft_consensus.cc:482] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:45:32.869529 31988 raft_consensus.cc:3037] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:32.871721 31988 raft_consensus.cc:504] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defdb5c159d4a898ac0050344109e60" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44119 } } peers { permanent_uuid: "b39aadc0af474fa2a8de3b6923da4526" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 44369 } } peers { permanent_uuid: "2c551fd550dc448cb842112faaff1d6d" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 39083 } }
I20260811 18:45:32.872306 31988 leader_election.cc:288] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526 [CANDIDATE]: Term 1 election: Requested vote from peers 1defdb5c159d4a898ac0050344109e60 (127.30.185.130:44119), 2c551fd550dc448cb842112faaff1d6d (127.30.185.129:39083)
I20260811 18:45:32.872769 31637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55e25afc90d049d384ce6cc4dcedc2e1" candidate_uuid: "b39aadc0af474fa2a8de3b6923da4526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1defdb5c159d4a898ac0050344109e60"
I20260811 18:45:32.872849 31395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55e25afc90d049d384ce6cc4dcedc2e1" candidate_uuid: "b39aadc0af474fa2a8de3b6923da4526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c551fd550dc448cb842112faaff1d6d"
I20260811 18:45:32.872964 31637 raft_consensus.cc:3037] T 55e25afc90d049d384ce6cc4dcedc2e1 P 1defdb5c159d4a898ac0050344109e60 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:32.873117 31395 raft_consensus.cc:3037] T 55e25afc90d049d384ce6cc4dcedc2e1 P 2c551fd550dc448cb842112faaff1d6d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:32.875278 31637 raft_consensus.cc:2454] T 55e25afc90d049d384ce6cc4dcedc2e1 P 1defdb5c159d4a898ac0050344109e60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b39aadc0af474fa2a8de3b6923da4526 in term 1.
I20260811 18:45:32.875676 31395 raft_consensus.cc:2454] T 55e25afc90d049d384ce6cc4dcedc2e1 P 2c551fd550dc448cb842112faaff1d6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b39aadc0af474fa2a8de3b6923da4526 in term 1.
I20260811 18:45:32.875815 31747 leader_election.cc:302] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1defdb5c159d4a898ac0050344109e60, b39aadc0af474fa2a8de3b6923da4526; no voters:
I20260811 18:45:32.876158 31988 raft_consensus.cc:2781] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:45:32.876652 31988 raft_consensus.cc:686] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526 [term 1 LEADER]: Becoming Leader. State: Replica: b39aadc0af474fa2a8de3b6923da4526, State: Running, Role: LEADER
I20260811 18:45:32.877161 31988 consensus_queue.cc:237] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defdb5c159d4a898ac0050344109e60" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44119 } } peers { permanent_uuid: "b39aadc0af474fa2a8de3b6923da4526" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 44369 } } peers { permanent_uuid: "2c551fd550dc448cb842112faaff1d6d" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 39083 } }
I20260811 18:45:32.882426 31152 catalog_manager.cc:5495] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526 reported cstate change: term changed from 0 to 1, leader changed from <none> to b39aadc0af474fa2a8de3b6923da4526 (127.30.185.131). New cstate: current_term: 1 leader_uuid: "b39aadc0af474fa2a8de3b6923da4526" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1defdb5c159d4a898ac0050344109e60" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b39aadc0af474fa2a8de3b6923da4526" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 44369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c551fd550dc448cb842112faaff1d6d" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 39083 } health_report { overall_health: UNKNOWN } } }
I20260811 18:45:32.901714 31972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:32.895513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51478 (local address 127.30.185.131:44369)
0811 18:45:32.895740 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:32.895746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:32.895867 (+ 121us) server_negotiation.cc:407] Connection header received
0811 18:45:32.896149 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:32.896154 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:32.896231 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:32.896389 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:45:32.896821 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.897651 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.899207 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.899633 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.899779 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:32.900366 (+ 587us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:45:32.901006 (+ 640us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:45:32.901179 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:32.901270 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 18:45:32.901355 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 18:45:32.910646 31637 raft_consensus.cc:1261] T 55e25afc90d049d384ce6cc4dcedc2e1 P 1defdb5c159d4a898ac0050344109e60 [term 1 FOLLOWER]: Refusing update from remote peer b39aadc0af474fa2a8de3b6923da4526: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:45:32.910645 31395 raft_consensus.cc:1261] T 55e25afc90d049d384ce6cc4dcedc2e1 P 2c551fd550dc448cb842112faaff1d6d [term 1 FOLLOWER]: Refusing update from remote peer b39aadc0af474fa2a8de3b6923da4526: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:45:32.911739 31988 consensus_queue.cc:1035] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c551fd550dc448cb842112faaff1d6d" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 39083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:45:32.912070 31993 consensus_queue.cc:1035] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1defdb5c159d4a898ac0050344109e60" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 44119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:45:32.929509 32004 mvcc.cc:205] Tried to move back new op lower bound from 7317397229190455296 to 7317397229068824576. Current Snapshot: MvccSnapshot[applied={T|T < 7317397229190455296}]
I20260811 18:45:32.942617 31152 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:45:32.943058 31152 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:45:32.946820 31152 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 846be1e75b9243fc8ca7a9e14b23e082: Sending DeleteTablet for 3 replicas of tablet 55e25afc90d049d384ce6cc4dcedc2e1
I20260811 18:45:32.948203 31843 tablet_service.cc:1514] Processing DeleteTablet for tablet 55e25afc90d049d384ce6cc4dcedc2e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:45:32 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:51462
I20260811 18:45:32.948194 31606 tablet_service.cc:1514] Processing DeleteTablet for tablet 55e25afc90d049d384ce6cc4dcedc2e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:45:32 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:36166
I20260811 18:45:32.949074 31354 tablet_service.cc:1514] Processing DeleteTablet for tablet 55e25afc90d049d384ce6cc4dcedc2e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:45:32 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:41736
I20260811 18:45:32.949693 32020 tablet_replica.cc:330] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526: stopping tablet replica
I20260811 18:45:32.950198 32020 raft_consensus.cc:2229] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:45:32.950959 32020 raft_consensus.cc:2258] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:45:32.950997 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 31255
I20260811 18:45:32.953259 32020 ts_tablet_manager.cc:1905] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:45:32.959033 32024 tablet_replica.cc:330] T 55e25afc90d049d384ce6cc4dcedc2e1 P 2c551fd550dc448cb842112faaff1d6d: stopping tablet replica
I20260811 18:45:32.960588 32024 raft_consensus.cc:2229] T 55e25afc90d049d384ce6cc4dcedc2e1 P 2c551fd550dc448cb842112faaff1d6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:45:32.961658 32020 ts_tablet_manager.cc:1918] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:45:32.961905 32020 log.cc:1196] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/55e25afc90d049d384ce6cc4dcedc2e1
I20260811 18:45:32.962598 32020 ts_tablet_manager.cc:1939] T 55e25afc90d049d384ce6cc4dcedc2e1 P b39aadc0af474fa2a8de3b6923da4526: Deleting consensus metadata
I20260811 18:45:32.962574 32024 raft_consensus.cc:2258] T 55e25afc90d049d384ce6cc4dcedc2e1 P 2c551fd550dc448cb842112faaff1d6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:45:32.964066 32022 tablet_replica.cc:330] T 55e25afc90d049d384ce6cc4dcedc2e1 P 1defdb5c159d4a898ac0050344109e60: stopping tablet replica
I20260811 18:45:32.964890 31135 catalog_manager.cc:4841] TS b39aadc0af474fa2a8de3b6923da4526 (127.30.185.131:44369): tablet 55e25afc90d049d384ce6cc4dcedc2e1 (table test-table [id=2e1739cf47424fb9b6bdfa5126880f83]) successfully deleted
I20260811 18:45:32.966066 32022 raft_consensus.cc:2229] T 55e25afc90d049d384ce6cc4dcedc2e1 P 1defdb5c159d4a898ac0050344109e60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:45:32.966699 32022 raft_consensus.cc:2258] T 55e25afc90d049d384ce6cc4dcedc2e1 P 1defdb5c159d4a898ac0050344109e60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:45:32.967252 32024 ts_tablet_manager.cc:1905] T 55e25afc90d049d384ce6cc4dcedc2e1 P 2c551fd550dc448cb842112faaff1d6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:45:32.969224 32022 ts_tablet_manager.cc:1905] T 55e25afc90d049d384ce6cc4dcedc2e1 P 1defdb5c159d4a898ac0050344109e60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:45:32.975138 32024 ts_tablet_manager.cc:1918] T 55e25afc90d049d384ce6cc4dcedc2e1 P 2c551fd550dc448cb842112faaff1d6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:45:32.975368 32024 log.cc:1196] T 55e25afc90d049d384ce6cc4dcedc2e1 P 2c551fd550dc448cb842112faaff1d6d: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-0/wal/wals/55e25afc90d049d384ce6cc4dcedc2e1
I20260811 18:45:32.975960 32024 ts_tablet_manager.cc:1939] T 55e25afc90d049d384ce6cc4dcedc2e1 P 2c551fd550dc448cb842112faaff1d6d: Deleting consensus metadata
I20260811 18:45:32.977243 32022 ts_tablet_manager.cc:1918] T 55e25afc90d049d384ce6cc4dcedc2e1 P 1defdb5c159d4a898ac0050344109e60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:45:32.977633 32022 log.cc:1196] T 55e25afc90d049d384ce6cc4dcedc2e1 P 1defdb5c159d4a898ac0050344109e60: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/55e25afc90d049d384ce6cc4dcedc2e1
I20260811 18:45:32.978112 31133 catalog_manager.cc:4841] TS 2c551fd550dc448cb842112faaff1d6d (127.30.185.129:39083): tablet 55e25afc90d049d384ce6cc4dcedc2e1 (table test-table [id=2e1739cf47424fb9b6bdfa5126880f83]) successfully deleted
I20260811 18:45:32.978422 32022 ts_tablet_manager.cc:1939] T 55e25afc90d049d384ce6cc4dcedc2e1 P 1defdb5c159d4a898ac0050344109e60: Deleting consensus metadata
Aug 11 18:45:32 dist-test-slave-z8x9 krb5kdc[31081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.129@KRBTEST.COM
I20260811 18:45:32.980170 31134 catalog_manager.cc:4841] TS 1defdb5c159d4a898ac0050344109e60 (127.30.185.130:44119): tablet 55e25afc90d049d384ce6cc4dcedc2e1 (table test-table [id=2e1739cf47424fb9b6bdfa5126880f83]) successfully deleted
I20260811 18:45:32.988469 31978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:32.953560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41746 (local address 127.30.185.129:39083)
0811 18:45:32.954331 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:32.954340 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:32.965753 (+ 11413us) server_negotiation.cc:407] Connection header received
0811 18:45:32.966116 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:32.966122 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:32.966219 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:32.966328 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:32.969523 (+ 3195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.970686 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.972738 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:32.973658 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:32.973770 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:32.980693 (+ 6923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:32.980707 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:32.980723 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:32.980769 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:32.984572 (+ 3803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:32.985734 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:32.985742 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:32.985745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:32.985923 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:32.986448 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:32.986453 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:32.986455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:32.986905 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:32.987149 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:32.987573 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 18:45:32.987786 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587}
I20260811 18:45:32.989516 31469 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:33.083036 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 31255
I20260811 18:45:33.119201 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 31490
Aug 11 18:45:33 dist-test-slave-z8x9 krb5kdc[31081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.130@KRBTEST.COM
I20260811 18:45:33.136875 31971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:33.120069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36170 (local address 127.30.185.130:44119)
0811 18:45:33.120279 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:33.120284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:33.120345 (+ 61us) server_negotiation.cc:407] Connection header received
0811 18:45:33.120551 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:33.120555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:33.120631 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:33.120712 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:33.122554 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:33.123484 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:33.125421 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:33.125800 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:33.125940 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:33.130722 (+ 4782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:33.130739 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:33.130756 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:33.130795 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:33.133931 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:33.134668 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:33.134676 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:33.134678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:33.134803 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:33.135199 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:33.135208 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:33.135210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:33.135629 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:33.135957 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:33.136271 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 18:45:33.136517 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 18:45:33.137609 31705 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:33.230881 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 31490
I20260811 18:45:33.256827 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 31721
Aug 11 18:45:33 dist-test-slave-z8x9 krb5kdc[31081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.131@KRBTEST.COM
I20260811 18:45:33.273968 31972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:33.257743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51492 (local address 127.30.185.131:44369)
0811 18:45:33.257937 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:33.257944 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:33.258020 (+ 76us) server_negotiation.cc:407] Connection header received
0811 18:45:33.258233 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:33.258239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:33.258327 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:33.258438 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:33.259826 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:33.260676 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:33.262191 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:33.262673 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:33.262837 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:33.267395 (+ 4558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:33.267407 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:33.267423 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:33.267455 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:33.270864 (+ 3409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:33.271621 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:33.271637 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:33.271640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:33.271789 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:33.272200 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:33.272207 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:33.272214 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:33.272606 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:33.272978 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:33.273266 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 18:45:33.273500 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 18:45:33.274781 31944 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:33.398394 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 31721
I20260811 18:45:33.428922 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 31102
I20260811 18:45:33.430320 31202 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:33.544039 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 31102
2026-08-11T18:45:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5043 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T18:45:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:45:33Z Disabled control of system clock
I20260811 18:45:33.657457 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:33649
--webserver_interface=127.30.185.190
--webserver_port=0
--builtin_ntp_servers=127.30.185.148:33213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:33649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:45:34.003304 32037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:34.003757 32037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:34.003897 32037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:34.011255 32037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:45:34.011379 32037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:34.011418 32037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:34.011476 32037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:45:34.011523 32037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:45:34.020979 32037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:33213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:33649
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:33649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:34.022363 32037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:34.024273 32037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:34.036504 32048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:34.036594 32049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:34.036556 32052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:34.037448 32037 server_base.cc:1034] running on GCE node
I20260811 18:45:34.038319 32037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:34.039812 32037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:34.041055 32037 hybrid_clock.cc:648] HybridClock initialized: now 1786473934041018 us; error 55 us; skew 500 ppm
I20260811 18:45:34.041492 32037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:45:34.043972 32037 webserver.cc:460] Webserver started at http://127.30.185.190:45079/ using document root <none> and password file <none>
I20260811 18:45:34.044720 32037 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:34.044865 32037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:34.045257 32037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:34.048388 32037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "7a3873692ce44728b9ddb6e2abc22832"
format_stamp: "Formatted at 2026-08-11 18:45:34 on dist-test-slave-z8x9"
server_key: "8338f2ef743e8d13eaeb121b8f344a16"
server_key_iv: "f7ba69b48bbdb38f235f927b479e9d7d"
server_key_version: "encryptionkey@0"
I20260811 18:45:34.049328 32037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "7a3873692ce44728b9ddb6e2abc22832"
format_stamp: "Formatted at 2026-08-11 18:45:34 on dist-test-slave-z8x9"
server_key: "8338f2ef743e8d13eaeb121b8f344a16"
server_key_iv: "f7ba69b48bbdb38f235f927b479e9d7d"
server_key_version: "encryptionkey@0"
I20260811 18:45:34.054648 32037 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 18:45:34.058312 32063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:34.059636 32037 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:45:34.059834 32037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "7a3873692ce44728b9ddb6e2abc22832"
format_stamp: "Formatted at 2026-08-11 18:45:34 on dist-test-slave-z8x9"
server_key: "8338f2ef743e8d13eaeb121b8f344a16"
server_key_iv: "f7ba69b48bbdb38f235f927b479e9d7d"
server_key_version: "encryptionkey@0"
I20260811 18:45:34.060060 32037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:34.077150 32037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:34.078091 32037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:34.078462 32037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:34.098546 32037 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:33649
I20260811 18:45:34.098595 32146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:33649 every 8 connection(s)
I20260811 18:45:34.099964 32037 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:45:34.103861 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:34.109328 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 32037
I20260811 18:45:34.109529 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:45:34.110013 31462 external_mini_cluster.cc:1428] Setting key a912d8c55e14a739c0c13831a51e603c
I20260811 18:45:34.115216 32147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832: Bootstrap starting.
I20260811 18:45:34.119014 32147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:34.120376 32147 log.cc:824] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:34.123898 32147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832: No bootstrap required, opened a new log
I20260811 18:45:34.125624 32150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:34.112095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55150 (local address 127.30.185.190:33649)
0811 18:45:34.113311 (+ 1216us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:34.113326 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:34.113370 (+ 44us) server_negotiation.cc:407] Connection header received
0811 18:45:34.115049 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:34.115072 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:34.115570 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:34.115902 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:34.116573 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:34.118025 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:34.120207 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:34.121140 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:34.121386 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:34.122565 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:34.122576 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:34.122596 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:34.123347 (+ 751us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:34.123459 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:34.123486 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:34.123594 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:34.123907 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 18:45:34.124296 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":439,"threads_started":1}
I20260811 18:45:34.132989 32147 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3873692ce44728b9ddb6e2abc22832" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33649 } }
I20260811 18:45:34.133441 32147 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:34.133528 32147 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a3873692ce44728b9ddb6e2abc22832, State: Initialized, Role: FOLLOWER
I20260811 18:45:34.134038 32147 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3873692ce44728b9ddb6e2abc22832" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33649 } }
I20260811 18:45:34.134243 32147 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:45:34.134375 32147 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:45:34.134536 32147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:34.137120 32147 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3873692ce44728b9ddb6e2abc22832" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33649 } }
I20260811 18:45:34.137733 32147 leader_election.cc:302] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a3873692ce44728b9ddb6e2abc22832; no voters:
I20260811 18:45:34.138391 32147 leader_election.cc:288] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:45:34.138587 32152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:45:34.139461 32152 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832 [term 1 LEADER]: Becoming Leader. State: Replica: 7a3873692ce44728b9ddb6e2abc22832, State: Running, Role: LEADER
I20260811 18:45:34.140033 32152 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3873692ce44728b9ddb6e2abc22832" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33649 } }
I20260811 18:45:34.140645 32147 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:45:34.143201 32156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a3873692ce44728b9ddb6e2abc22832. Latest consensus state: current_term: 1 leader_uuid: "7a3873692ce44728b9ddb6e2abc22832" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3873692ce44728b9ddb6e2abc22832" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33649 } } }
I20260811 18:45:34.143432 32153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a3873692ce44728b9ddb6e2abc22832" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a3873692ce44728b9ddb6e2abc22832" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33649 } } }
I20260811 18:45:34.143720 32156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832 [sys.catalog]: This master's current role is: LEADER
I20260811 18:45:34.143740 32153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832 [sys.catalog]: This master's current role is: LEADER
I20260811 18:45:34.145769 32160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:45:34.151213 32160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:45:34.160557 32160 catalog_manager.cc:1347] Generated new cluster ID: 72d25b9c876b4dd0aa7d4a2d5ddf84db
I20260811 18:45:34.160707 32160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:45:34.183964 32160 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:45:34.185947 32160 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:45:34.199832 32160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832: Generated new TSK 0
I20260811 18:45:34.200974 32160 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:45:34.209879 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:33649
--builtin_ntp_servers=127.30.185.148:33213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:45:34.566370 32188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:34.566671 32188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:34.566764 32188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:34.574172 32188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:34.574286 32188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:34.574396 32188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:45:34.583858 32188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:33213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:33649
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:34.585124 32188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:34.586928 32188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:34.600428 32197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:34.600279 32199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:34.600349 32202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:34.601373 32188 server_base.cc:1034] running on GCE node
I20260811 18:45:34.602033 32188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:34.603008 32188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:34.604259 32188 hybrid_clock.cc:648] HybridClock initialized: now 1786473934604214 us; error 51 us; skew 500 ppm
I20260811 18:45:34.604591 32188 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:45:34.607215 32188 webserver.cc:460] Webserver started at http://127.30.185.129:34377/ using document root <none> and password file <none>
I20260811 18:45:34.607950 32188 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:34.608088 32188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:34.608435 32188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:34.611387 32188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "34350c18bef24a1c9abdadd645819547"
format_stamp: "Formatted at 2026-08-11 18:45:34 on dist-test-slave-z8x9"
server_key: "985c332adf00cba8af9a9ec662123cc2"
server_key_iv: "c3f118ad44ea6a6c308b2cfdb3f7f445"
server_key_version: "encryptionkey@0"
I20260811 18:45:34.612190 32188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "34350c18bef24a1c9abdadd645819547"
format_stamp: "Formatted at 2026-08-11 18:45:34 on dist-test-slave-z8x9"
server_key: "985c332adf00cba8af9a9ec662123cc2"
server_key_iv: "c3f118ad44ea6a6c308b2cfdb3f7f445"
server_key_version: "encryptionkey@0"
I20260811 18:45:34.617457 32188 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 18:45:34.620833 32213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:34.622229 32188 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:45:34.622435 32188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "34350c18bef24a1c9abdadd645819547"
format_stamp: "Formatted at 2026-08-11 18:45:34 on dist-test-slave-z8x9"
server_key: "985c332adf00cba8af9a9ec662123cc2"
server_key_iv: "c3f118ad44ea6a6c308b2cfdb3f7f445"
server_key_version: "encryptionkey@0"
I20260811 18:45:34.622664 32188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:34.647174 32188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:34.648044 32188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:34.648381 32188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:34.649472 32188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:45:34.650980 32188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:45:34.651109 32188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:34.651244 32188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:45:34.651342 32188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:34.700809 32188 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:33181
I20260811 18:45:34.700924 32391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:33181 every 8 connection(s)
I20260811 18:45:34.702786 32188 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:45:34.707662 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 32188
I20260811 18:45:34.707871 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:45:34.708477 31462 external_mini_cluster.cc:1428] Setting key b2761900f52ae18285b0b4ec483816e8
I20260811 18:45:34.712010 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:33649
--builtin_ntp_servers=127.30.185.148:33213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 18:45:34.721975 32394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:34.709322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:38213 (local address 127.30.185.190:33649)
0811 18:45:34.709844 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:34.709851 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:34.711196 (+ 1345us) server_negotiation.cc:407] Connection header received
0811 18:45:34.714100 (+ 2904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:34.714106 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:34.714187 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:34.714312 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:34.715725 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:34.716458 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:34.718967 (+ 2509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:34.719376 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:34.719478 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:34.721141 (+ 1663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:34.721146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:34.721149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:34.721224 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:34.721284 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:34.721299 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:34.721372 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:34.721636 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 18:45:34.721716 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":241,"threads_started":1}
I20260811 18:45:34.723507 32397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:34.709825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:33649 (local address 127.30.185.129:38213)
0811 18:45:34.711017 (+ 1192us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:34.711078 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:34.713791 (+ 2713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:34.714553 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:34.714567 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:34.714693 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:45:34.715528 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:34.715556 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:34.716633 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:34.716640 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:34.718807 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:34.718825 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:34.719605 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:34.719615 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:34.719976 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:34.720763 (+ 787us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:45:34.720769 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:45:34.720825 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:45:34.720939 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:45:34.721399 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:45:34.721414 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:45:34.721424 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 18:45:34.721620 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 18:45:34.722059 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":373,"threads_started":1}
I20260811 18:45:34.725273 32392 heartbeater.cc:347] Connected to a master server at 127.30.185.190:33649
I20260811 18:45:34.725597 32392 heartbeater.cc:464] Registering TS with master...
I20260811 18:45:34.726502 32392 heartbeater.cc:511] Master 127.30.185.190:33649 requested a full tablet report, sending...
I20260811 18:45:34.728689 32088 ts_manager.cc:194] Registered new tserver with Master: 34350c18bef24a1c9abdadd645819547 (127.30.185.129:33181)
I20260811 18:45:34.730268 32088 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.185.129:38213
W20260811 18:45:35.078191 32399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:35.078511 32399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:35.078680 32399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:35.086932 32399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:35.087059 32399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:35.087256 32399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:45:35.097235 32399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:33213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:33649
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:35.098664 32399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:35.100451 32399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:35.115590 32408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:35.115535 32409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:35.116485 32412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:35.117539 32399 server_base.cc:1034] running on GCE node
I20260811 18:45:35.118172 32399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:35.119120 32399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:35.120388 32399 hybrid_clock.cc:648] HybridClock initialized: now 1786473935120301 us; error 61 us; skew 500 ppm
I20260811 18:45:35.120813 32399 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:45:35.123405 32399 webserver.cc:460] Webserver started at http://127.30.185.130:36927/ using document root <none> and password file <none>
I20260811 18:45:35.124208 32399 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:35.124341 32399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:35.124748 32399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:35.128113 32399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "43310ef8cfe04e85b840f0b79137918d"
format_stamp: "Formatted at 2026-08-11 18:45:35 on dist-test-slave-z8x9"
server_key: "d21383e1c652c16856534f8439098de9"
server_key_iv: "9694bdc702ca94eeca755d95a267dcc8"
server_key_version: "encryptionkey@0"
I20260811 18:45:35.128939 32399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "43310ef8cfe04e85b840f0b79137918d"
format_stamp: "Formatted at 2026-08-11 18:45:35 on dist-test-slave-z8x9"
server_key: "d21383e1c652c16856534f8439098de9"
server_key_iv: "9694bdc702ca94eeca755d95a267dcc8"
server_key_version: "encryptionkey@0"
I20260811 18:45:35.134272 32399 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 18:45:35.138377 32424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:35.139873 32399 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:45:35.140092 32399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "43310ef8cfe04e85b840f0b79137918d"
format_stamp: "Formatted at 2026-08-11 18:45:35 on dist-test-slave-z8x9"
server_key: "d21383e1c652c16856534f8439098de9"
server_key_iv: "9694bdc702ca94eeca755d95a267dcc8"
server_key_version: "encryptionkey@0"
I20260811 18:45:35.140355 32399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:35.157513 32399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:35.158545 32399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:35.158908 32399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:35.160020 32399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:45:35.161717 32399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:45:35.161860 32399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:35.162024 32399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:45:35.162144 32399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:35.198766 32399 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:42295
I20260811 18:45:35.198802 32624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:42295 every 8 connection(s)
I20260811 18:45:35.200354 32399 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:45:35.209111 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 32399
I20260811 18:45:35.209358 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:45:35.209837 31462 external_mini_cluster.cc:1428] Setting key f839a9cbec78eb427c7965ae1323a7c3
I20260811 18:45:35.212881 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:33649
--builtin_ntp_servers=127.30.185.148:33213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 18:45:35.214116 32394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:35.203096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:56013 (local address 127.30.185.190:33649)
0811 18:45:35.203301 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:35.203305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:35.204762 (+ 1457us) server_negotiation.cc:407] Connection header received
0811 18:45:35.206560 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:35.206565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:35.206637 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:35.206734 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:35.207902 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.208910 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.210817 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.211449 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.211580 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:35.213225 (+ 1645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:35.213236 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:35.213239 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:35.213258 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:35.213296 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:35.213310 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:35.213362 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:35.213670 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 18:45:35.213779 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":56}
I20260811 18:45:35.215122 32628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:35.203621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:33649 (local address 127.30.185.130:56013)
0811 18:45:35.204564 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:35.204608 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:35.206252 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:35.206943 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:35.206950 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:35.207035 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:45:35.207632 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:35.207648 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.209078 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.209082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:35.210643 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:35.210659 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.211699 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.211706 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:35.211936 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:35.212668 (+ 732us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:45:35.212670 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:45:35.212753 (+ 83us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:45:35.212912 (+ 159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:45:35.213463 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:45:35.213472 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:45:35.213477 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:45:35.213656 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 18:45:35.214067 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":288,"threads_started":1}
I20260811 18:45:35.216563 32625 heartbeater.cc:347] Connected to a master server at 127.30.185.190:33649
I20260811 18:45:35.216897 32625 heartbeater.cc:464] Registering TS with master...
I20260811 18:45:35.217756 32625 heartbeater.cc:511] Master 127.30.185.190:33649 requested a full tablet report, sending...
I20260811 18:45:35.219364 32088 ts_manager.cc:194] Registered new tserver with Master: 43310ef8cfe04e85b840f0b79137918d (127.30.185.130:42295)
I20260811 18:45:35.220615 32088 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.185.130:56013
W20260811 18:45:35.560223 32629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:35.560518 32629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:35.560671 32629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:35.568214 32629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:35.568334 32629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:35.568511 32629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:45:35.578140 32629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:33213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:33649
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:35.579461 32629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:35.581252 32629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:35.595178 32641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:35.595256 32639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:35.595276 32636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:35.596149 32629 server_base.cc:1034] running on GCE node
I20260811 18:45:35.596875 32629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:35.597890 32629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:35.599138 32629 hybrid_clock.cc:648] HybridClock initialized: now 1786473935599089 us; error 56 us; skew 500 ppm
I20260811 18:45:35.599471 32629 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:45:35.601872 32629 webserver.cc:460] Webserver started at http://127.30.185.131:36809/ using document root <none> and password file <none>
I20260811 18:45:35.602586 32629 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:35.602725 32629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:35.603055 32629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:35.606115 32629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "dd7c9e25c7ec452888cdfe116b988cae"
format_stamp: "Formatted at 2026-08-11 18:45:35 on dist-test-slave-z8x9"
server_key: "da84e8461b631ea33f0760af56af6677"
server_key_iv: "52e10c5e1686c1251b722a6c8c50854b"
server_key_version: "encryptionkey@0"
I20260811 18:45:35.606954 32629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "dd7c9e25c7ec452888cdfe116b988cae"
format_stamp: "Formatted at 2026-08-11 18:45:35 on dist-test-slave-z8x9"
server_key: "da84e8461b631ea33f0760af56af6677"
server_key_iv: "52e10c5e1686c1251b722a6c8c50854b"
server_key_version: "encryptionkey@0"
I20260811 18:45:35.612170 32629 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:45:35.615598 32651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:35.616911 32629 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:45:35.617086 32629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "dd7c9e25c7ec452888cdfe116b988cae"
format_stamp: "Formatted at 2026-08-11 18:45:35 on dist-test-slave-z8x9"
server_key: "da84e8461b631ea33f0760af56af6677"
server_key_iv: "52e10c5e1686c1251b722a6c8c50854b"
server_key_version: "encryptionkey@0"
I20260811 18:45:35.617388 32629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:35.640002 32629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:35.640933 32629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:35.641280 32629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:35.642365 32629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:45:35.643885 32629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:45:35.644011 32629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:35.644155 32629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:45:35.644219 32629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:35.678354 32629 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:42369
I20260811 18:45:35.678452 373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:42369 every 8 connection(s)
I20260811 18:45:35.679670 32629 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:45:35.687392 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 32629
I20260811 18:45:35.687594 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:45:35.688052 31462 external_mini_cluster.cc:1428] Setting key f0aec26c31493489152d4a857c854c5d
I20260811 18:45:35.692444 32394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:35.682352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:57473 (local address 127.30.185.190:33649)
0811 18:45:35.682532 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:35.682537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:35.683835 (+ 1298us) server_negotiation.cc:407] Connection header received
0811 18:45:35.685693 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:35.685699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:35.685791 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:35.685905 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:35.686922 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.687761 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.689747 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.690286 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.690384 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:35.691803 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:35.691807 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:35.691810 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:35.691827 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:35.691860 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:35.691872 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:35.691941 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:35.692128 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 18:45:35.692203 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:45:35.693439 376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:35.682825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:33649 (local address 127.30.185.131:57473)
0811 18:45:35.683671 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:35.683707 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:35.685429 (+ 1722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:35.686120 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:35.686127 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:35.686208 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:45:35.686744 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:35.686760 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.687917 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.687921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:35.689586 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:35.689598 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.690510 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.690517 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:35.690826 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:35.691514 (+ 688us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:45:35.691516 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:45:35.691554 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:45:35.691630 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:45:35.691954 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:45:35.691962 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:45:35.691967 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:45:35.692099 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 18:45:35.692366 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":262,"threads_started":1}
I20260811 18:45:35.694646 374 heartbeater.cc:347] Connected to a master server at 127.30.185.190:33649
I20260811 18:45:35.694954 374 heartbeater.cc:464] Registering TS with master...
I20260811 18:45:35.695668 374 heartbeater.cc:511] Master 127.30.185.190:33649 requested a full tablet report, sending...
I20260811 18:45:35.696987 32088 ts_manager.cc:194] Registered new tserver with Master: dd7c9e25c7ec452888cdfe116b988cae (127.30.185.131:42369)
I20260811 18:45:35.697825 32088 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.185.131:57473
I20260811 18:45:35.703285 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:45:35.713279 32394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:35.707415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55152 (local address 127.30.185.190:33649)
0811 18:45:35.707610 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:35.707614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:35.708100 (+ 486us) server_negotiation.cc:407] Connection header received
0811 18:45:35.708322 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:35.708326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:35.708383 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:35.708466 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:35.709091 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.709863 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.711123 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.711537 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.711627 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:35.712680 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:35.712684 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:35.712688 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:35.712704 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:35.712732 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:35.712743 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:35.712808 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:35.712928 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 18:45:35.712997 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:45:35.717074 32088 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55152:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:45:35.724143 32088 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:45:35.734081 32392 heartbeater.cc:503] Master 127.30.185.190:33649 was elected leader, sending a full tablet report...
I20260811 18:45:35.757513 402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:35.737027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43774 (local address 127.30.185.129:33181)
0811 18:45:35.744425 (+ 7398us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:35.744439 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:35.744898 (+ 459us) server_negotiation.cc:407] Connection header received
0811 18:45:35.745323 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:35.745333 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:35.745652 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:35.745875 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:35.747148 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.749391 (+ 2243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.753289 (+ 3898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.755343 (+ 2054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.755563 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:35.755876 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:35.756752 (+ 876us) server_negotiation.cc:299] Negotiation successful
0811 18:45:35.757000 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7152,"thread_start_us":6985,"threads_started":1}
I20260811 18:45:35.758441 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:35.736324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:33181 (local address 127.0.0.1:43774)
0811 18:45:35.744769 (+ 8445us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:35.744790 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:35.745068 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:35.746318 (+ 1250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:35.746330 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:35.746371 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:35.746913 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:35.746930 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.749655 (+ 2725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.749663 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:35.752954 (+ 3291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:35.752976 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.756097 (+ 3121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.756106 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:35.756408 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:35.756482 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 18:45:35.757816 (+ 1334us) client_negotiation.cc:240] Negotiation successful
0811 18:45:35.758059 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8294,"thread_start_us":975,"threads_started":1}
I20260811 18:45:35.759235 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:35.736164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:42369 (local address 127.0.0.1:36834)
0811 18:45:35.737261 (+ 1097us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:35.737318 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:35.737533 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:35.746979 (+ 9446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:35.746987 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:35.747024 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:35.747547 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:35.747563 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.751822 (+ 4259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.751831 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:35.754785 (+ 2954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:35.754807 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.758462 (+ 3655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.758469 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:35.758659 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:35.758691 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 18:45:35.758759 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 18:45:35.758906 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"thread_start_us":8278,"threads_started":1}
I20260811 18:45:35.759882 400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:35.736495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36834 (local address 127.30.185.131:42369)
0811 18:45:35.746258 (+ 9763us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:35.746278 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:35.746325 (+ 47us) server_negotiation.cc:407] Connection header received
0811 18:45:35.746442 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:35.746451 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:35.746704 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:35.747805 (+ 1101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:35.748175 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.751627 (+ 3452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.756450 (+ 4823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.758190 (+ 1740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.759087 (+ 897us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:35.759293 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:35.759370 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 18:45:35.759541 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9517,"thread_start_us":8156,"threads_started":1}
I20260811 18:45:35.762856 32735 tablet_service.cc:1467] Processing CreateTablet for tablet dbd53cc79be2423d91a4c7d989516989 (DEFAULT_TABLE table=test-table [id=acbcf5232b95447fb76da57760aa3953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:45:35.763141 32286 tablet_service.cc:1467] Processing CreateTablet for tablet dbd53cc79be2423d91a4c7d989516989 (DEFAULT_TABLE table=test-table [id=acbcf5232b95447fb76da57760aa3953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:45:35.764427 32735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd53cc79be2423d91a4c7d989516989. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:35.764602 32286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd53cc79be2423d91a4c7d989516989. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:35.766317 406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:35.745140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:42295 (local address 127.0.0.1:60430)
0811 18:45:35.758130 (+ 12990us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:35.758150 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:35.758279 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:35.759152 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:35.759157 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:35.759183 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:35.759567 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:35.759584 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.761509 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.761514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:35.763520 (+ 2006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:35.763536 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.765529 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.765535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:35.765712 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:35.765745 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 18:45:35.765865 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 18:45:35.766001 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12870,"thread_start_us":12706,"threads_started":1}
I20260811 18:45:35.766467 408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:35.745246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60430 (local address 127.30.185.130:42295)
0811 18:45:35.757530 (+ 12284us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:35.757541 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:35.758277 (+ 736us) server_negotiation.cc:407] Connection header received
0811 18:45:35.758436 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:35.758446 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:35.758693 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:35.758914 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:35.759788 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.761330 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.763748 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.765325 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.765493 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:35.765701 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:35.765929 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 18:45:35.766097 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12091,"thread_start_us":12291,"threads_started":1}
I20260811 18:45:35.772972 32504 tablet_service.cc:1467] Processing CreateTablet for tablet dbd53cc79be2423d91a4c7d989516989 (DEFAULT_TABLE table=test-table [id=acbcf5232b95447fb76da57760aa3953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:45:35.774475 32504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd53cc79be2423d91a4c7d989516989. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:35.776554 409 tablet_bootstrap.cc:492] T dbd53cc79be2423d91a4c7d989516989 P 34350c18bef24a1c9abdadd645819547: Bootstrap starting.
I20260811 18:45:35.778585 411 tablet_bootstrap.cc:492] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae: Bootstrap starting.
I20260811 18:45:35.781435 411 tablet_bootstrap.cc:654] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:35.781636 409 tablet_bootstrap.cc:654] T dbd53cc79be2423d91a4c7d989516989 P 34350c18bef24a1c9abdadd645819547: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:35.782498 411 log.cc:824] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:35.782842 409 log.cc:824] T dbd53cc79be2423d91a4c7d989516989 P 34350c18bef24a1c9abdadd645819547: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:35.785559 409 tablet_bootstrap.cc:492] T dbd53cc79be2423d91a4c7d989516989 P 34350c18bef24a1c9abdadd645819547: No bootstrap required, opened a new log
I20260811 18:45:35.785796 411 tablet_bootstrap.cc:492] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae: No bootstrap required, opened a new log
I20260811 18:45:35.785876 409 ts_tablet_manager.cc:1397] T dbd53cc79be2423d91a4c7d989516989 P 34350c18bef24a1c9abdadd645819547: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260811 18:45:35.786059 411 ts_tablet_manager.cc:1397] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260811 18:45:35.790220 417 tablet_bootstrap.cc:492] T dbd53cc79be2423d91a4c7d989516989 P 43310ef8cfe04e85b840f0b79137918d: Bootstrap starting.
I20260811 18:45:35.793998 417 tablet_bootstrap.cc:654] T dbd53cc79be2423d91a4c7d989516989 P 43310ef8cfe04e85b840f0b79137918d: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:35.794656 411 raft_consensus.cc:348] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7c9e25c7ec452888cdfe116b988cae" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42369 } } peers { permanent_uuid: "34350c18bef24a1c9abdadd645819547" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 33181 } } peers { permanent_uuid: "43310ef8cfe04e85b840f0b79137918d" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 42295 } }
I20260811 18:45:35.795028 411 raft_consensus.cc:374] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:35.795164 411 raft_consensus.cc:729] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7c9e25c7ec452888cdfe116b988cae, State: Initialized, Role: FOLLOWER
I20260811 18:45:35.795243 417 log.cc:824] T dbd53cc79be2423d91a4c7d989516989 P 43310ef8cfe04e85b840f0b79137918d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:35.795805 411 consensus_queue.cc:260] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7c9e25c7ec452888cdfe116b988cae" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42369 } } peers { permanent_uuid: "34350c18bef24a1c9abdadd645819547" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 33181 } } peers { permanent_uuid: "43310ef8cfe04e85b840f0b79137918d" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 42295 } }
I20260811 18:45:35.797945 374 heartbeater.cc:503] Master 127.30.185.190:33649 was elected leader, sending a full tablet report...
I20260811 18:45:35.797873 409 raft_consensus.cc:348] T dbd53cc79be2423d91a4c7d989516989 P 34350c18bef24a1c9abdadd645819547 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7c9e25c7ec452888cdfe116b988cae" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42369 } } peers { permanent_uuid: "34350c18bef24a1c9abdadd645819547" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 33181 } } peers { permanent_uuid: "43310ef8cfe04e85b840f0b79137918d" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 42295 } }
I20260811 18:45:35.798385 409 raft_consensus.cc:374] T dbd53cc79be2423d91a4c7d989516989 P 34350c18bef24a1c9abdadd645819547 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:35.798506 409 raft_consensus.cc:729] T dbd53cc79be2423d91a4c7d989516989 P 34350c18bef24a1c9abdadd645819547 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34350c18bef24a1c9abdadd645819547, State: Initialized, Role: FOLLOWER
I20260811 18:45:35.798637 411 ts_tablet_manager.cc:1428] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260811 18:45:35.799167 409 consensus_queue.cc:260] T dbd53cc79be2423d91a4c7d989516989 P 34350c18bef24a1c9abdadd645819547 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7c9e25c7ec452888cdfe116b988cae" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42369 } } peers { permanent_uuid: "34350c18bef24a1c9abdadd645819547" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 33181 } } peers { permanent_uuid: "43310ef8cfe04e85b840f0b79137918d" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 42295 } }
I20260811 18:45:35.800906 409 ts_tablet_manager.cc:1428] T dbd53cc79be2423d91a4c7d989516989 P 34350c18bef24a1c9abdadd645819547: Time spent starting tablet: real 0.015s user 0.012s sys 0.001s
I20260811 18:45:35.801918 417 tablet_bootstrap.cc:492] T dbd53cc79be2423d91a4c7d989516989 P 43310ef8cfe04e85b840f0b79137918d: No bootstrap required, opened a new log
I20260811 18:45:35.802217 417 ts_tablet_manager.cc:1397] T dbd53cc79be2423d91a4c7d989516989 P 43310ef8cfe04e85b840f0b79137918d: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20260811 18:45:35.810091 417 raft_consensus.cc:348] T dbd53cc79be2423d91a4c7d989516989 P 43310ef8cfe04e85b840f0b79137918d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7c9e25c7ec452888cdfe116b988cae" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42369 } } peers { permanent_uuid: "34350c18bef24a1c9abdadd645819547" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 33181 } } peers { permanent_uuid: "43310ef8cfe04e85b840f0b79137918d" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 42295 } }
I20260811 18:45:35.810472 417 raft_consensus.cc:374] T dbd53cc79be2423d91a4c7d989516989 P 43310ef8cfe04e85b840f0b79137918d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:35.810547 417 raft_consensus.cc:729] T dbd53cc79be2423d91a4c7d989516989 P 43310ef8cfe04e85b840f0b79137918d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43310ef8cfe04e85b840f0b79137918d, State: Initialized, Role: FOLLOWER
I20260811 18:45:35.811111 417 consensus_queue.cc:260] T dbd53cc79be2423d91a4c7d989516989 P 43310ef8cfe04e85b840f0b79137918d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7c9e25c7ec452888cdfe116b988cae" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42369 } } peers { permanent_uuid: "34350c18bef24a1c9abdadd645819547" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 33181 } } peers { permanent_uuid: "43310ef8cfe04e85b840f0b79137918d" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 42295 } }
I20260811 18:45:35.812135 32625 heartbeater.cc:503] Master 127.30.185.190:33649 was elected leader, sending a full tablet report...
I20260811 18:45:35.812853 417 ts_tablet_manager.cc:1428] T dbd53cc79be2423d91a4c7d989516989 P 43310ef8cfe04e85b840f0b79137918d: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260811 18:45:35.867389 420 raft_consensus.cc:482] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:45:35.867714 420 raft_consensus.cc:504] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7c9e25c7ec452888cdfe116b988cae" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42369 } } peers { permanent_uuid: "34350c18bef24a1c9abdadd645819547" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 33181 } } peers { permanent_uuid: "43310ef8cfe04e85b840f0b79137918d" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 42295 } }
I20260811 18:45:35.869519 420 leader_election.cc:288] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34350c18bef24a1c9abdadd645819547 (127.30.185.129:33181), 43310ef8cfe04e85b840f0b79137918d (127.30.185.130:42295)
I20260811 18:45:35.876946 402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:35.870126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:37233 (local address 127.30.185.129:33181)
0811 18:45:35.870386 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:35.870392 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:35.870412 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:45:35.870488 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:35.870496 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:35.870574 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:35.870652 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:35.871197 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.872195 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.874769 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.875993 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.876162 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:35.876249 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:35.876578 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 18:45:35.876703 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 18:45:35.876977 408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:35.870051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:44551 (local address 127.30.185.130:42295)
0811 18:45:35.870268 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:35.870276 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:35.870747 (+ 471us) server_negotiation.cc:407] Connection header received
0811 18:45:35.870883 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:35.870887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:35.870945 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:35.871017 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:35.871577 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.872509 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.874979 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.876150 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.876289 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:35.876342 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:35.876626 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 18:45:35.876739 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 18:45:35.877010 376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:35.869734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:33181 (local address 127.30.185.131:37233)
0811 18:45:35.869938 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:35.869961 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:35.870077 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:35.870733 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:35.870737 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:35.870810 (+ 73us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:35.871015 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:35.871027 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.872340 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.872343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:35.874608 (+ 2265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:35.874624 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.876158 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.876162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:35.876373 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:35.876415 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 18:45:35.876526 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 18:45:35.876670 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"mutex_wait_us":74,"spinlock_wait_cycles":19840}
I20260811 18:45:35.877257 427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:35.869845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:42295 (local address 127.30.185.131:44551)
0811 18:45:35.870603 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:35.870616 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:35.870760 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:35.871063 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:35.871068 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:35.871097 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:35.871417 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:35.871425 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.872648 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.872653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:35.874808 (+ 2155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:35.874818 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.876297 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.876304 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:35.876467 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:35.876509 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 18:45:35.876587 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 18:45:35.876679 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":466,"threads_started":1}
I20260811 18:45:35.878129 32320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd53cc79be2423d91a4c7d989516989" candidate_uuid: "dd7c9e25c7ec452888cdfe116b988cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34350c18bef24a1c9abdadd645819547" is_pre_election: true
I20260811 18:45:35.878185 32553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd53cc79be2423d91a4c7d989516989" candidate_uuid: "dd7c9e25c7ec452888cdfe116b988cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43310ef8cfe04e85b840f0b79137918d" is_pre_election: true
I20260811 18:45:35.878676 32320 raft_consensus.cc:2454] T dbd53cc79be2423d91a4c7d989516989 P 34350c18bef24a1c9abdadd645819547 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7c9e25c7ec452888cdfe116b988cae in term 0.
I20260811 18:45:35.878715 32553 raft_consensus.cc:2454] T dbd53cc79be2423d91a4c7d989516989 P 43310ef8cfe04e85b840f0b79137918d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7c9e25c7ec452888cdfe116b988cae in term 0.
I20260811 18:45:35.879392 32658 leader_election.cc:302] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 34350c18bef24a1c9abdadd645819547, 43310ef8cfe04e85b840f0b79137918d, dd7c9e25c7ec452888cdfe116b988cae; no voters:
I20260811 18:45:35.879771 420 raft_consensus.cc:2781] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:45:35.879915 420 raft_consensus.cc:482] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:45:35.880039 420 raft_consensus.cc:3037] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:35.882385 420 raft_consensus.cc:504] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7c9e25c7ec452888cdfe116b988cae" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42369 } } peers { permanent_uuid: "34350c18bef24a1c9abdadd645819547" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 33181 } } peers { permanent_uuid: "43310ef8cfe04e85b840f0b79137918d" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 42295 } }
I20260811 18:45:35.882967 420 leader_election.cc:288] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae [CANDIDATE]: Term 1 election: Requested vote from peers 34350c18bef24a1c9abdadd645819547 (127.30.185.129:33181), 43310ef8cfe04e85b840f0b79137918d (127.30.185.130:42295)
I20260811 18:45:35.883456 32553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd53cc79be2423d91a4c7d989516989" candidate_uuid: "dd7c9e25c7ec452888cdfe116b988cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43310ef8cfe04e85b840f0b79137918d"
I20260811 18:45:35.883474 32320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd53cc79be2423d91a4c7d989516989" candidate_uuid: "dd7c9e25c7ec452888cdfe116b988cae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34350c18bef24a1c9abdadd645819547"
I20260811 18:45:35.883664 32553 raft_consensus.cc:3037] T dbd53cc79be2423d91a4c7d989516989 P 43310ef8cfe04e85b840f0b79137918d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:35.883677 32320 raft_consensus.cc:3037] T dbd53cc79be2423d91a4c7d989516989 P 34350c18bef24a1c9abdadd645819547 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:35.886016 32320 raft_consensus.cc:2454] T dbd53cc79be2423d91a4c7d989516989 P 34350c18bef24a1c9abdadd645819547 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7c9e25c7ec452888cdfe116b988cae in term 1.
I20260811 18:45:35.886023 32553 raft_consensus.cc:2454] T dbd53cc79be2423d91a4c7d989516989 P 43310ef8cfe04e85b840f0b79137918d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7c9e25c7ec452888cdfe116b988cae in term 1.
I20260811 18:45:35.886453 32658 leader_election.cc:302] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34350c18bef24a1c9abdadd645819547, dd7c9e25c7ec452888cdfe116b988cae; no voters:
I20260811 18:45:35.886763 420 raft_consensus.cc:2781] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:45:35.887249 420 raft_consensus.cc:686] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae [term 1 LEADER]: Becoming Leader. State: Replica: dd7c9e25c7ec452888cdfe116b988cae, State: Running, Role: LEADER
I20260811 18:45:35.887733 420 consensus_queue.cc:237] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7c9e25c7ec452888cdfe116b988cae" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42369 } } peers { permanent_uuid: "34350c18bef24a1c9abdadd645819547" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 33181 } } peers { permanent_uuid: "43310ef8cfe04e85b840f0b79137918d" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 42295 } }
I20260811 18:45:35.892558 32085 catalog_manager.cc:5495] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7c9e25c7ec452888cdfe116b988cae (127.30.185.131). New cstate: current_term: 1 leader_uuid: "dd7c9e25c7ec452888cdfe116b988cae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7c9e25c7ec452888cdfe116b988cae" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34350c18bef24a1c9abdadd645819547" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 33181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43310ef8cfe04e85b840f0b79137918d" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 42295 } health_report { overall_health: UNKNOWN } } }
W20260811 18:45:35.931504 375 tablet.cc:2369] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:45:35.942974 400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:35.937522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36848 (local address 127.30.185.131:42369)
0811 18:45:35.937723 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:35.937728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:35.937746 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:45:35.937905 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:35.937910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:35.937997 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:35.938131 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:45:35.938505 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.939239 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.940710 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:35.941141 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:35.941311 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:35.941884 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:45:35.942411 (+ 527us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:45:35.942524 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:35.942601 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 18:45:35.942666 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 18:45:35.951843 32553 raft_consensus.cc:1261] T dbd53cc79be2423d91a4c7d989516989 P 43310ef8cfe04e85b840f0b79137918d [term 1 FOLLOWER]: Refusing update from remote peer dd7c9e25c7ec452888cdfe116b988cae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:45:35.951881 32320 raft_consensus.cc:1261] T dbd53cc79be2423d91a4c7d989516989 P 34350c18bef24a1c9abdadd645819547 [term 1 FOLLOWER]: Refusing update from remote peer dd7c9e25c7ec452888cdfe116b988cae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260811 18:45:35.952651 32627 tablet.cc:2369] T dbd53cc79be2423d91a4c7d989516989 P 43310ef8cfe04e85b840f0b79137918d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:45:35.952869 428 consensus_queue.cc:1035] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "34350c18bef24a1c9abdadd645819547" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 33181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:45:35.953272 420 consensus_queue.cc:1035] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae [LEADER]: Connected to new peer: Peer: permanent_uuid: "43310ef8cfe04e85b840f0b79137918d" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 42295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:45:35.958602 32393 tablet.cc:2369] T dbd53cc79be2423d91a4c7d989516989 P 34350c18bef24a1c9abdadd645819547: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:45:35.965808 431 mvcc.cc:205] Tried to move back new op lower bound from 7317397241649004544 to 7317397241399881728. Current Snapshot: MvccSnapshot[applied={T|T < 7317397241649004544}]
I20260811 18:45:35.982285 32088 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:45:35.982633 32088 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:45:35.986442 32088 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7a3873692ce44728b9ddb6e2abc22832: Sending DeleteTablet for 3 replicas of tablet dbd53cc79be2423d91a4c7d989516989
I20260811 18:45:35.987668 32735 tablet_service.cc:1514] Processing DeleteTablet for tablet dbd53cc79be2423d91a4c7d989516989 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:45:35 UTC) from {username='slave'} at 127.0.0.1:36834
I20260811 18:45:35.987701 32504 tablet_service.cc:1514] Processing DeleteTablet for tablet dbd53cc79be2423d91a4c7d989516989 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:45:35 UTC) from {username='slave'} at 127.0.0.1:60430
I20260811 18:45:35.987646 32286 tablet_service.cc:1514] Processing DeleteTablet for tablet dbd53cc79be2423d91a4c7d989516989 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:45:35 UTC) from {username='slave'} at 127.0.0.1:43774
I20260811 18:45:35.990367 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 32188
I20260811 18:45:36.003288 456 tablet_replica.cc:330] T dbd53cc79be2423d91a4c7d989516989 P 34350c18bef24a1c9abdadd645819547: stopping tablet replica
I20260811 18:45:36.004066 456 raft_consensus.cc:2229] T dbd53cc79be2423d91a4c7d989516989 P 34350c18bef24a1c9abdadd645819547 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:45:36.005014 456 raft_consensus.cc:2258] T dbd53cc79be2423d91a4c7d989516989 P 34350c18bef24a1c9abdadd645819547 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:45:36.006225 453 tablet_replica.cc:330] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae: stopping tablet replica
I20260811 18:45:36.006995 455 tablet_replica.cc:330] T dbd53cc79be2423d91a4c7d989516989 P 43310ef8cfe04e85b840f0b79137918d: stopping tablet replica
I20260811 18:45:36.007489 455 raft_consensus.cc:2229] T dbd53cc79be2423d91a4c7d989516989 P 43310ef8cfe04e85b840f0b79137918d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:45:36.008061 455 raft_consensus.cc:2258] T dbd53cc79be2423d91a4c7d989516989 P 43310ef8cfe04e85b840f0b79137918d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:45:36.008873 456 ts_tablet_manager.cc:1905] T dbd53cc79be2423d91a4c7d989516989 P 34350c18bef24a1c9abdadd645819547: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:45:36.010326 455 ts_tablet_manager.cc:1905] T dbd53cc79be2423d91a4c7d989516989 P 43310ef8cfe04e85b840f0b79137918d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:45:36.010895 453 raft_consensus.cc:2229] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:45:36.012475 402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:35.992165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43788 (local address 127.30.185.129:33181)
0811 18:45:35.992369 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:35.992375 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:36.002912 (+ 10537us) server_negotiation.cc:407] Connection header received
0811 18:45:36.003169 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:36.003178 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:36.003319 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:36.003439 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:36.004145 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:36.005506 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:36.007662 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:36.008339 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:36.008449 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:36.010592 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:36.010604 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:36.010622 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:36.011427 (+ 805us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:36.011495 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:36.011533 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:36.011626 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:36.011906 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 18:45:36.012032 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 18:45:36.013377 453 raft_consensus.cc:2258] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:45:36.013720 32385 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:36.017480 453 ts_tablet_manager.cc:1905] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:45:36.017534 455 ts_tablet_manager.cc:1918] T dbd53cc79be2423d91a4c7d989516989 P 43310ef8cfe04e85b840f0b79137918d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:45:36.017695 455 log.cc:1196] T dbd53cc79be2423d91a4c7d989516989 P 43310ef8cfe04e85b840f0b79137918d: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/dbd53cc79be2423d91a4c7d989516989
I20260811 18:45:36.018313 455 ts_tablet_manager.cc:1939] T dbd53cc79be2423d91a4c7d989516989 P 43310ef8cfe04e85b840f0b79137918d: Deleting consensus metadata
I20260811 18:45:36.020287 32065 catalog_manager.cc:4841] TS 43310ef8cfe04e85b840f0b79137918d (127.30.185.130:42295): tablet dbd53cc79be2423d91a4c7d989516989 (table test-table [id=acbcf5232b95447fb76da57760aa3953]) successfully deleted
I20260811 18:45:36.025944 453 ts_tablet_manager.cc:1918] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:45:36.026122 453 log.cc:1196] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/dbd53cc79be2423d91a4c7d989516989
I20260811 18:45:36.026690 453 ts_tablet_manager.cc:1939] T dbd53cc79be2423d91a4c7d989516989 P dd7c9e25c7ec452888cdfe116b988cae: Deleting consensus metadata
I20260811 18:45:36.028585 32065 catalog_manager.cc:4841] TS dd7c9e25c7ec452888cdfe116b988cae (127.30.185.131:42369): tablet dbd53cc79be2423d91a4c7d989516989 (table test-table [id=acbcf5232b95447fb76da57760aa3953]) successfully deleted
I20260811 18:45:36.114511 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 32188
I20260811 18:45:36.147568 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 32399
W20260811 18:45:36.147971 32067 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.185.129:33181 (error 108)
I20260811 18:45:36.159264 408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:36.149646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60436 (local address 127.30.185.130:42295)
0811 18:45:36.149836 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:36.149841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:36.149860 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:45:36.149929 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:36.149936 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:36.150048 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:36.150148 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:36.150794 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:36.151946 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:36.153976 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:36.154622 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:36.154745 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:36.156931 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:36.156944 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:36.156964 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:36.157828 (+ 864us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:36.157895 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:36.157922 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:36.158044 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:36.158731 (+ 687us) server_negotiation.cc:299] Negotiation successful
0811 18:45:36.158859 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 18:45:36.160030 406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:36.156134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:33181 (local address 127.0.0.1:43794)
0811 18:45:36.159658 (+ 3524us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:36.159844 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3360}
I20260811 18:45:36.160089 32619 generic_service.cc:196] Checking for leaks (request via RPC)
W20260811 18:45:36.160319 32067 catalog_manager.cc:4584] TS 34350c18bef24a1c9abdadd645819547 (127.30.185.129:33181): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dbd53cc79be2423d91a4c7d989516989: Network error: Client connection negotiation failed: client connection to 127.30.185.129:33181: connect: Connection refused (error 111)
I20260811 18:45:36.214313 406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:36.213690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:33181 (local address 127.0.0.1:43798)
0811 18:45:36.213991 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:36.214105 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260811 18:45:36.215561 406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:36.215101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:33181 (local address 127.0.0.1:43804)
0811 18:45:36.215287 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:36.215382 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 18:45:36.269230 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 32399
I20260811 18:45:36.276288 406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:36.275790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:33181 (local address 127.0.0.1:43818)
0811 18:45:36.275994 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:36.276108 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 18:45:36.277343 406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:36.276936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:33181 (local address 127.0.0.1:43824)
0811 18:45:36.277083 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:36.277217 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 18:45:36.301926 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 32629
I20260811 18:45:36.309985 400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:36.302778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36856 (local address 127.30.185.131:42369)
0811 18:45:36.302968 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:36.302973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:36.303043 (+ 70us) server_negotiation.cc:407] Connection header received
0811 18:45:36.303237 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:36.303242 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:36.303333 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:36.303422 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:36.303931 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:36.304683 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:36.306645 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:36.307083 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:36.307179 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:36.308414 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:36.308423 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:36.308442 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:36.309060 (+ 618us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:36.309107 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:36.309127 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:36.309312 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:36.309450 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 18:45:36.309544 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 18:45:36.310746 367 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:36.358048 406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:36.357479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:33181 (local address 127.0.0.1:43840)
0811 18:45:36.357747 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:36.357858 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260811 18:45:36.359239 406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:36.358828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:33181 (local address 127.0.0.1:43852)
0811 18:45:36.359048 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:36.359132 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260811 18:45:36.438529 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 32629
I20260811 18:45:36.469602 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 32037
I20260811 18:45:36.470795 32139 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:36.581314 406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:36.580582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:33181 (local address 127.0.0.1:43868)
0811 18:45:36.580844 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:36.580982 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 18:45:36.581511 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 32037
2026-08-11T18:45:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3023 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:45:36 dist-test-slave-z8x9 krb5kdc[474](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:45:36 dist-test-slave-z8x9 krb5kdc[474](info): set up 2 sockets
Aug 11 18:45:36 dist-test-slave-z8x9 krb5kdc[474](info): commencing operation
krb5kdc: starting...
W20260811 18:45:38.015115 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.385s user 0.001s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:45:38 dist-test-slave-z8x9 krb5kdc[474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:45:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:45:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:45:38.364285 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:38489
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:35871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:38489
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:45:38.725222 493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:38.725550 493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:38.725735 493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:38.733071 493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:45:38.733208 493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:38.733259 493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:38.733381 493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:45:38.733469 493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:45:38.743862 493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:35871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:38489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:38489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:38.745379 493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:38.746804 493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:38.759085 501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:38.759177 506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:38.759253 500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:38.760121 493 server_base.cc:1034] running on GCE node
I20260811 18:45:38.761294 493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:38.762761 493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:38.763988 493 hybrid_clock.cc:648] HybridClock initialized: now 1786473938763941 us; error 38 us; skew 500 ppm
I20260811 18:45:38.764375 493 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:45:38 dist-test-slave-z8x9 krb5kdc[474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473938, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:45:38.767650 493 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:45:38.769263 493 webserver.cc:460] Webserver started at http://127.30.185.190:36953/ using document root <none> and password file <none>
I20260811 18:45:38.769979 493 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:38.770139 493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:38.770462 493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:38.773622 493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "864c5fa9c2bb43a3844bc51849be666f"
format_stamp: "Formatted at 2026-08-11 18:45:38 on dist-test-slave-z8x9"
server_key: "2be7fb7decea8661c886feab48787442"
server_key_iv: "fbc8d340cc0e576a133ec0855fb48d22"
server_key_version: "encryptionkey@0"
I20260811 18:45:38.774390 493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "864c5fa9c2bb43a3844bc51849be666f"
format_stamp: "Formatted at 2026-08-11 18:45:38 on dist-test-slave-z8x9"
server_key: "2be7fb7decea8661c886feab48787442"
server_key_iv: "fbc8d340cc0e576a133ec0855fb48d22"
server_key_version: "encryptionkey@0"
I20260811 18:45:38.779606 493 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260811 18:45:38.783195 517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:38.784494 493 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 18:45:38.784691 493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "864c5fa9c2bb43a3844bc51849be666f"
format_stamp: "Formatted at 2026-08-11 18:45:38 on dist-test-slave-z8x9"
server_key: "2be7fb7decea8661c886feab48787442"
server_key_iv: "fbc8d340cc0e576a133ec0855fb48d22"
server_key_version: "encryptionkey@0"
I20260811 18:45:38.784914 493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:38.815004 493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:38.819347 493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:38.819669 493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:38.839074 493 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:38489
I20260811 18:45:38.839078 610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:38489 every 8 connection(s)
I20260811 18:45:38.840495 493 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:45:38.844288 611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:38.849447 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 493
I20260811 18:45:38.849634 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:45:38.850099 31462 external_mini_cluster.cc:1428] Setting key 01cdd157c6c0ac4be2acd48162525e68
I20260811 18:45:38.855355 611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f: Bootstrap starting.
I20260811 18:45:38.859167 611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:38.860327 611 log.cc:824] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:38.863358 611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f: No bootstrap required, opened a new log
Aug 11 18:45:38 dist-test-slave-z8x9 krb5kdc[474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:45:38.872912 611 raft_consensus.cc:348] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864c5fa9c2bb43a3844bc51849be666f" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 38489 } }
I20260811 18:45:38.873509 611 raft_consensus.cc:374] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:38.873636 611 raft_consensus.cc:729] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 864c5fa9c2bb43a3844bc51849be666f, State: Initialized, Role: FOLLOWER
I20260811 18:45:38.874248 611 consensus_queue.cc:260] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864c5fa9c2bb43a3844bc51849be666f" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 38489 } }
I20260811 18:45:38.874512 611 raft_consensus.cc:388] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:45:38.874680 611 raft_consensus.cc:482] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:45:38.874786 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:38.852024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56830 (local address 127.30.185.190:38489)
0811 18:45:38.853321 (+ 1297us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:38.853336 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:38.853379 (+ 43us) server_negotiation.cc:407] Connection header received
0811 18:45:38.855098 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:38.855121 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:38.855559 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:38.855904 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:38.857996 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:38.859378 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:38.860828 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:38.861612 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:38.861821 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:38.866754 (+ 4933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:38.866762 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:38.866777 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:38.866816 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:38.870184 (+ 3368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:38.871128 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:38.871134 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:38.871140 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:38.871317 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:38.871729 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:38.871732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:38.871734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:38.872281 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:38.872490 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:38.873011 (+ 521us) server_negotiation.cc:299] Negotiation successful
0811 18:45:38.873396 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1022,"thread_start_us":773,"threads_started":1}
I20260811 18:45:38.874881 611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:38.877357 611 raft_consensus.cc:504] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864c5fa9c2bb43a3844bc51849be666f" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 38489 } }
I20260811 18:45:38.877923 611 leader_election.cc:302] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 864c5fa9c2bb43a3844bc51849be666f; no voters:
I20260811 18:45:38.878459 611 leader_election.cc:288] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:45:38.878609 617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:45:38.879343 617 raft_consensus.cc:686] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f [term 1 LEADER]: Becoming Leader. State: Replica: 864c5fa9c2bb43a3844bc51849be666f, State: Running, Role: LEADER
I20260811 18:45:38.879848 617 consensus_queue.cc:237] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864c5fa9c2bb43a3844bc51849be666f" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 38489 } }
I20260811 18:45:38.880810 611 sys_catalog.cc:564] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:45:38.883073 619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 864c5fa9c2bb43a3844bc51849be666f. Latest consensus state: current_term: 1 leader_uuid: "864c5fa9c2bb43a3844bc51849be666f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864c5fa9c2bb43a3844bc51849be666f" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 38489 } } }
I20260811 18:45:38.883546 619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f [sys.catalog]: This master's current role is: LEADER
I20260811 18:45:38.883181 618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "864c5fa9c2bb43a3844bc51849be666f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "864c5fa9c2bb43a3844bc51849be666f" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 38489 } } }
I20260811 18:45:38.889089 618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f [sys.catalog]: This master's current role is: LEADER
I20260811 18:45:38.893926 630 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:45:38.898447 630 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:45:38.906463 630 catalog_manager.cc:1347] Generated new cluster ID: 9389f4107c5b483ca16f255d163cf73d
I20260811 18:45:38.906589 630 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:45:38.926406 630 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:45:38.928413 630 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:45:38.946689 630 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f: Generated new TSK 0
I20260811 18:45:38.947554 630 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.185.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
I20260811 18:45:39.044504 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:38489
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:35871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:45:39.423724 660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:39.424024 660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:39.424201 660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:39.431681 660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:39.431798 660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:39.432029 660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:45:39.441951 660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:35871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:38489
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:39.443307 660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:39.444650 660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:39.458316 666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:39.458195 668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:39.458272 670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:39.459378 660 server_base.cc:1034] running on GCE node
I20260811 18:45:39.460091 660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:39.461135 660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:39.462436 660 hybrid_clock.cc:648] HybridClock initialized: now 1786473939462370 us; error 58 us; skew 500 ppm
I20260811 18:45:39.462818 660 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:45:39 dist-test-slave-z8x9 krb5kdc[474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473939, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:45:39.466148 660 init.cc:377] Logged in from keytab as kudu/127.30.185.129@KRBTEST.COM (short username kudu)
I20260811 18:45:39.467753 660 webserver.cc:460] Webserver started at http://127.30.185.129:45977/ using document root <none> and password file <none>
I20260811 18:45:39.468478 660 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:39.468633 660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:39.468966 660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:39.472033 660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "f25a656a146f4edd961abc7c98b3b88e"
format_stamp: "Formatted at 2026-08-11 18:45:39 on dist-test-slave-z8x9"
server_key: "0c647b118cc4e725b3c2be06b870b7c6"
server_key_iv: "266b337bc6ecf96d1228f30cf5d5e2eb"
server_key_version: "encryptionkey@0"
I20260811 18:45:39.472837 660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "f25a656a146f4edd961abc7c98b3b88e"
format_stamp: "Formatted at 2026-08-11 18:45:39 on dist-test-slave-z8x9"
server_key: "0c647b118cc4e725b3c2be06b870b7c6"
server_key_iv: "266b337bc6ecf96d1228f30cf5d5e2eb"
server_key_version: "encryptionkey@0"
I20260811 18:45:39.478370 660 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 18:45:39.482023 683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:39.483361 660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:45:39.483547 660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "f25a656a146f4edd961abc7c98b3b88e"
format_stamp: "Formatted at 2026-08-11 18:45:39 on dist-test-slave-z8x9"
server_key: "0c647b118cc4e725b3c2be06b870b7c6"
server_key_iv: "266b337bc6ecf96d1228f30cf5d5e2eb"
server_key_version: "encryptionkey@0"
I20260811 18:45:39.483753 660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:39.494899 660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:39.499094 660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:39.499409 660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:39.500387 660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:45:39.501744 660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:45:39.501852 660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:39.502012 660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:45:39.502087 660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:39.536480 660 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:33269
I20260811 18:45:39.536571 870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:33269 every 8 connection(s)
I20260811 18:45:39.537834 660 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:45:39.544529 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 660
I20260811 18:45:39.544724 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:45:39.545329 31462 external_mini_cluster.cc:1428] Setting key 264e513ba6eecd0f99e8942c925a9dec
Aug 11 18:45:39 dist-test-slave-z8x9 krb5kdc[474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473939, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:45:39.560061 875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:39.540472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:57795 (local address 127.30.185.190:38489)
0811 18:45:39.541006 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:39.541013 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:39.541897 (+ 884us) server_negotiation.cc:407] Connection header received
0811 18:45:39.543665 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:39.543669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:39.543741 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:39.543857 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:39.545944 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:39.546875 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:39.548623 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:39.549400 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:39.549553 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:39.552928 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:39.552937 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:39.552945 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:39.552989 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:39.556680 (+ 3691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:39.557467 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:39.557475 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:39.557481 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:39.557635 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:39.558071 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:39.558078 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:39.558083 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:39.558479 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:39.558645 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:39.559330 (+ 685us) server_negotiation.cc:299] Negotiation successful
0811 18:45:39.559587 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":235,"threads_started":1}
I20260811 18:45:39.560741 876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:39.540953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:38489 (local address 127.30.185.129:57795)
0811 18:45:39.541711 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:39.541756 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:39.543409 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:39.544055 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:39.544088 (+ 33us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:39.544911 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:45:39.545732 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:39.545759 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:39.547045 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:39.547050 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:39.548464 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:39.548476 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:39.549610 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:39.549615 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:39.549815 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:39.550530 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:45:39.550552 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:45:39.552698 (+ 2146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:45:39.556832 (+ 4134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:45:39.556838 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:45:39.556849 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:45:39.557352 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:45:39.557749 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:45:39.557752 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:45:39.557754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:45:39.557951 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:45:39.558607 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:45:39.558613 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:45:39.558928 (+ 315us) client_negotiation.cc:769] Sending connection context
0811 18:45:39.559155 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 18:45:39.559532 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":300,"threads_started":1}
I20260811 18:45:39.562357 871 heartbeater.cc:347] Connected to a master server at 127.30.185.190:38489
I20260811 18:45:39.562678 871 heartbeater.cc:464] Registering TS with master...
I20260811 18:45:39.563376 871 heartbeater.cc:511] Master 127.30.185.190:38489 requested a full tablet report, sending...
I20260811 18:45:39.565521 548 ts_manager.cc:194] Registered new tserver with Master: f25a656a146f4edd961abc7c98b3b88e (127.30.185.129:33269)
I20260811 18:45:39.566983 548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:57795
WARNING: no policy specified for kudu/127.30.185.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
I20260811 18:45:39.617746 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:38489
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:35871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:45:39.965719 880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:39.966038 880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:39.966136 880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:39.973716 880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:39.973841 880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:39.973956 880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:45:39.984086 880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:35871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:38489
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:39.985567 880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:39.986961 880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:40.000820 889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:40.000841 887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:40.000874 886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:40.001747 880 server_base.cc:1034] running on GCE node
I20260811 18:45:40.002374 880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:40.003345 880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:40.004573 880 hybrid_clock.cc:648] HybridClock initialized: now 1786473940004526 us; error 40 us; skew 500 ppm
I20260811 18:45:40.004906 880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:45:40 dist-test-slave-z8x9 krb5kdc[474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473940, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:45:40.008224 880 init.cc:377] Logged in from keytab as kudu/127.30.185.130@KRBTEST.COM (short username kudu)
I20260811 18:45:40.009836 880 webserver.cc:460] Webserver started at http://127.30.185.130:43563/ using document root <none> and password file <none>
I20260811 18:45:40.010548 880 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:40.010684 880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:40.011024 880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:40.014047 880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "010dcbdf7ba54fa1955ef6e13326129d"
format_stamp: "Formatted at 2026-08-11 18:45:40 on dist-test-slave-z8x9"
server_key: "3dfaf1718c82ccf7687b7a7dcb89e02b"
server_key_iv: "33bd6bfb664137771dd29055beb93c45"
server_key_version: "encryptionkey@0"
I20260811 18:45:40.014837 880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "010dcbdf7ba54fa1955ef6e13326129d"
format_stamp: "Formatted at 2026-08-11 18:45:40 on dist-test-slave-z8x9"
server_key: "3dfaf1718c82ccf7687b7a7dcb89e02b"
server_key_iv: "33bd6bfb664137771dd29055beb93c45"
server_key_version: "encryptionkey@0"
I20260811 18:45:40.020264 880 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:45:40.023792 904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:40.025076 880 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:45:40.025326 880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "010dcbdf7ba54fa1955ef6e13326129d"
format_stamp: "Formatted at 2026-08-11 18:45:40 on dist-test-slave-z8x9"
server_key: "3dfaf1718c82ccf7687b7a7dcb89e02b"
server_key_iv: "33bd6bfb664137771dd29055beb93c45"
server_key_version: "encryptionkey@0"
I20260811 18:45:40.025552 880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:40.044914 880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:40.049845 880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:40.050182 880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:40.051179 880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:45:40.052560 880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:45:40.052670 880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:40.052856 880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:45:40.052938 880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:40.085858 880 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:43877
I20260811 18:45:40.085935 1084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:43877 every 8 connection(s)
I20260811 18:45:40.087393 880 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:45:40.091758 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 880
I20260811 18:45:40.091990 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:45:40.092435 31462 external_mini_cluster.cc:1428] Setting key 17d0db5ba6a8e6dd42515057e1a3ca01
Aug 11 18:45:40 dist-test-slave-z8x9 krb5kdc[474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473940, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:45:40.108937 1088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:40.090231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:37221 (local address 127.30.185.190:38489)
0811 18:45:40.090722 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:40.090727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:40.091787 (+ 1060us) server_negotiation.cc:407] Connection header received
0811 18:45:40.094130 (+ 2343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:40.094135 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:40.094199 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:40.094296 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:40.096016 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.096732 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.098400 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.098850 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.098992 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:40.102780 (+ 3788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:40.102786 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:40.102789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:40.102825 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:40.105644 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:40.106484 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:40.106496 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:40.106499 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:40.106636 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:40.107107 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:40.107111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:40.107113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:40.107384 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:40.107560 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:40.108288 (+ 728us) server_negotiation.cc:299] Negotiation successful
0811 18:45:40.108524 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":245,"threads_started":1}
I20260811 18:45:40.110342 1089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:40.090838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:38489 (local address 127.30.185.130:37221)
0811 18:45:40.091606 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:40.091645 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:40.093812 (+ 2167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:40.094625 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:40.094638 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:40.095242 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:45:40.095823 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:40.095843 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.096906 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.096911 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:40.098244 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:40.098257 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.099095 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.099102 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:40.099357 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:40.100158 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:45:40.100187 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:45:40.102522 (+ 2335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:45:40.105798 (+ 3276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:45:40.105808 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:45:40.105826 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:45:40.106293 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:45:40.106757 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:45:40.106762 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:45:40.106766 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:45:40.106974 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:45:40.107520 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:45:40.107532 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:45:40.107909 (+ 377us) client_negotiation.cc:769] Sending connection context
0811 18:45:40.108184 (+ 275us) client_negotiation.cc:240] Negotiation successful
0811 18:45:40.108769 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":309,"threads_started":1}
I20260811 18:45:40.111860 1086 heartbeater.cc:347] Connected to a master server at 127.30.185.190:38489
I20260811 18:45:40.112182 1086 heartbeater.cc:464] Registering TS with master...
I20260811 18:45:40.112920 1086 heartbeater.cc:511] Master 127.30.185.190:38489 requested a full tablet report, sending...
I20260811 18:45:40.114444 548 ts_manager.cc:194] Registered new tserver with Master: 010dcbdf7ba54fa1955ef6e13326129d (127.30.185.130:43877)
I20260811 18:45:40.115290 548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:37221
WARNING: no policy specified for kudu/127.30.185.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
I20260811 18:45:40.169637 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:38489
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:35871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:45:40.517372 1098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:40.517658 1098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:40.517755 1098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:40.524981 1098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:40.525094 1098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:40.525276 1098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:45:40.535112 1098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:35871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:38489
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:40.536438 1098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:40.537834 1098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:40.553252 1109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:40.553252 1106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:40.553480 1103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:40.554320 1098 server_base.cc:1034] running on GCE node
I20260811 18:45:40.555002 1098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:40.556012 1098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:40.557317 1098 hybrid_clock.cc:648] HybridClock initialized: now 1786473940557217 us; error 62 us; skew 500 ppm
I20260811 18:45:40.557653 1098 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:45:40 dist-test-slave-z8x9 krb5kdc[474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473940, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:45:40.560890 1098 init.cc:377] Logged in from keytab as kudu/127.30.185.131@KRBTEST.COM (short username kudu)
I20260811 18:45:40.562477 1098 webserver.cc:460] Webserver started at http://127.30.185.131:39855/ using document root <none> and password file <none>
I20260811 18:45:40.563187 1098 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:40.563337 1098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:40.563653 1098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:40.566928 1098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "2b175bd28c5d420ebb309f74c631c031"
format_stamp: "Formatted at 2026-08-11 18:45:40 on dist-test-slave-z8x9"
server_key: "8234d373ed25a7cc84901fb5e86ec8a8"
server_key_iv: "4176a97f80a47ae2101d884f8226d940"
server_key_version: "encryptionkey@0"
I20260811 18:45:40.567783 1098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "2b175bd28c5d420ebb309f74c631c031"
format_stamp: "Formatted at 2026-08-11 18:45:40 on dist-test-slave-z8x9"
server_key: "8234d373ed25a7cc84901fb5e86ec8a8"
server_key_iv: "4176a97f80a47ae2101d884f8226d940"
server_key_version: "encryptionkey@0"
I20260811 18:45:40.569931 871 heartbeater.cc:503] Master 127.30.185.190:38489 was elected leader, sending a full tablet report...
I20260811 18:45:40.573477 1098 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 18:45:40.576905 1121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:40.578227 1098 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:45:40.578408 1098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "2b175bd28c5d420ebb309f74c631c031"
format_stamp: "Formatted at 2026-08-11 18:45:40 on dist-test-slave-z8x9"
server_key: "8234d373ed25a7cc84901fb5e86ec8a8"
server_key_iv: "4176a97f80a47ae2101d884f8226d940"
server_key_version: "encryptionkey@0"
I20260811 18:45:40.578611 1098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:40.592898 1098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:40.597422 1098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:40.597733 1098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:40.598740 1098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:45:40.600128 1098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:45:40.600248 1098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:40.600406 1098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:45:40.600487 1098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:40.634946 1098 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:43895
I20260811 18:45:40.635001 1313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:43895 every 8 connection(s)
I20260811 18:45:40.636281 1098 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:45:40.644824 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 1098
I20260811 18:45:40.645052 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:45:40.645553 31462 external_mini_cluster.cc:1428] Setting key a81ef959c70f8de6aeba359fc244e282
Aug 11 18:45:40 dist-test-slave-z8x9 krb5kdc[474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473940, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:45:40.656872 1316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:40.638970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:50393 (local address 127.30.185.190:38489)
0811 18:45:40.639530 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:40.639535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:40.640302 (+ 767us) server_negotiation.cc:407] Connection header received
0811 18:45:40.642272 (+ 1970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:40.642278 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:40.642388 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:40.642514 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:40.644042 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.644819 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.646513 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.647030 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.647131 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:40.650932 (+ 3801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:40.650936 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:40.650940 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:40.650972 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:40.653647 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:40.654429 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:40.654439 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:40.654441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:40.654570 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:40.655021 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:40.655031 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:40.655033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:40.655312 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:40.655473 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:40.656255 (+ 782us) server_negotiation.cc:299] Negotiation successful
0811 18:45:40.656489 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":315,"threads_started":1}
I20260811 18:45:40.657944 1318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:40.639433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:38489 (local address 127.30.185.131:50393)
0811 18:45:40.640132 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:40.640176 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:40.641989 (+ 1813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:40.642757 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:40.642765 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:40.643299 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:45:40.643864 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:40.643881 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.644992 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.644996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:40.646369 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:40.646381 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.647266 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.647276 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:40.647492 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:40.648241 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:45:40.648275 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:45:40.650717 (+ 2442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:45:40.653792 (+ 3075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:45:40.653799 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:45:40.653812 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:45:40.654282 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:45:40.654692 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:45:40.654696 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:45:40.654705 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:45:40.654894 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:45:40.655443 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:45:40.655452 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:45:40.655821 (+ 369us) client_negotiation.cc:769] Sending connection context
0811 18:45:40.656082 (+ 261us) client_negotiation.cc:240] Negotiation successful
0811 18:45:40.656628 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":282,"threads_started":1}
I20260811 18:45:40.659461 1314 heartbeater.cc:347] Connected to a master server at 127.30.185.190:38489
I20260811 18:45:40.659842 1314 heartbeater.cc:464] Registering TS with master...
I20260811 18:45:40.660810 1314 heartbeater.cc:511] Master 127.30.185.190:38489 requested a full tablet report, sending...
I20260811 18:45:40.662639 548 ts_manager.cc:194] Registered new tserver with Master: 2b175bd28c5d420ebb309f74c631c031 (127.30.185.131:43895)
I20260811 18:45:40.663935 548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:50393
I20260811 18:45:40.673805 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:45:40.692391 1316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:40.677902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56832 (local address 127.30.185.190:38489)
0811 18:45:40.678076 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:40.678081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:40.678576 (+ 495us) server_negotiation.cc:407] Connection header received
0811 18:45:40.678790 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:40.678794 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:40.678882 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:40.678988 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:40.680466 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.681357 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.682959 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.683449 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.683584 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:40.686516 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:40.686522 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:40.686525 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:40.686560 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:40.689457 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:40.690241 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:40.690253 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:40.690255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:40.690377 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:40.690866 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:40.690870 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:40.690872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:40.691189 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:40.691481 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:40.691868 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 18:45:40.692080 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 18:45:40.696238 548 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:45:40.703109 548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:45:40.734750 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:40.715789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48782 (local address 127.30.185.129:33269)
0811 18:45:40.722241 (+ 6452us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:40.722253 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:40.722285 (+ 32us) server_negotiation.cc:407] Connection header received
0811 18:45:40.722412 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:40.722424 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:40.722748 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:40.722994 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:40.725655 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.727794 (+ 2139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.731477 (+ 3683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.733084 (+ 1607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.733311 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:40.733519 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:40.734175 (+ 656us) server_negotiation.cc:299] Negotiation successful
0811 18:45:40.734347 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6257,"thread_start_us":9088,"threads_started":1}
I20260811 18:45:40.735689 1335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:40.715142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:33269 (local address 127.0.0.1:48782)
0811 18:45:40.721465 (+ 6323us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:40.721489 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:40.721641 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:40.723353 (+ 1712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:40.723365 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:40.724788 (+ 1423us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:40.725377 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:40.725397 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.728466 (+ 3069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.728470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:40.731168 (+ 2698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:40.731183 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.733659 (+ 2476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.733669 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:40.733902 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:40.733965 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 18:45:40.735043 (+ 1078us) client_negotiation.cc:240] Negotiation successful
0811 18:45:40.735251 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6152,"thread_start_us":5970,"threads_started":1}
I20260811 18:45:40.735862 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:40.721998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54640 (local address 127.30.185.131:43895)
0811 18:45:40.728507 (+ 6509us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:40.728516 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:40.728549 (+ 33us) server_negotiation.cc:407] Connection header received
0811 18:45:40.728643 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:40.728650 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:40.728886 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:40.729090 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:40.729936 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.731151 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.733164 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.734478 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.734700 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:40.734998 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:40.735253 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 18:45:40.735399 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6346,"thread_start_us":6187,"threads_started":1}
I20260811 18:45:40.736023 1340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:40.721916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:43895 (local address 127.0.0.1:54640)
0811 18:45:40.727510 (+ 5594us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:40.727531 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:40.727649 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:40.729201 (+ 1552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:40.729208 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:40.729242 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:40.729700 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:40.729710 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.731298 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.731302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:40.732950 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:40.732963 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.734699 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.734705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:40.734874 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:40.735089 (+ 215us) client_negotiation.cc:769] Sending connection context
0811 18:45:40.735201 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 18:45:40.735337 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5461,"mutex_wait_us":59,"thread_start_us":5283,"threads_started":1}
I20260811 18:45:40.736464 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:40.715066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:43877 (local address 127.0.0.1:59034)
0811 18:45:40.716263 (+ 1197us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:40.716312 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:40.716529 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:40.728409 (+ 11880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:40.728414 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:40.728458 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:40.728940 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:40.728954 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.731173 (+ 2219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.731180 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:40.733693 (+ 2513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:40.733706 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.735628 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.735640 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:40.735862 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:40.735893 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 18:45:40.735964 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 18:45:40.736097 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":882,"mutex_wait_us":74,"thread_start_us":741,"threads_started":1}
I20260811 18:45:40.737123 1333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:40.715233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59034 (local address 127.30.185.130:43877)
0811 18:45:40.727787 (+ 12554us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:40.727796 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:40.727823 (+ 27us) server_negotiation.cc:407] Connection header received
0811 18:45:40.727919 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:40.727928 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:40.728184 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:40.729206 (+ 1022us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:40.729459 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.731023 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.733889 (+ 2866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.735379 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.736440 (+ 1061us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:40.736603 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:40.736675 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 18:45:40.736833 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12355,"thread_start_us":12467,"threads_started":1}
I20260811 18:45:40.739553 764 tablet_service.cc:1467] Processing CreateTablet for tablet bd51a72030254a1ea9d24fde63810a90 (DEFAULT_TABLE table=test-table [id=53dc7308b2e044bbb8cce169d891bc00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:45:40.739558 1215 tablet_service.cc:1467] Processing CreateTablet for tablet bd51a72030254a1ea9d24fde63810a90 (DEFAULT_TABLE table=test-table [id=53dc7308b2e044bbb8cce169d891bc00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:45:40.740996 764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd51a72030254a1ea9d24fde63810a90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:40.741099 1215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd51a72030254a1ea9d24fde63810a90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:40.742507 988 tablet_service.cc:1467] Processing CreateTablet for tablet bd51a72030254a1ea9d24fde63810a90 (DEFAULT_TABLE table=test-table [id=53dc7308b2e044bbb8cce169d891bc00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:45:40.743835 988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd51a72030254a1ea9d24fde63810a90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:40.754266 1342 tablet_bootstrap.cc:492] T bd51a72030254a1ea9d24fde63810a90 P 2b175bd28c5d420ebb309f74c631c031: Bootstrap starting.
I20260811 18:45:40.755292 1344 tablet_bootstrap.cc:492] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e: Bootstrap starting.
I20260811 18:45:40.759110 1344 tablet_bootstrap.cc:654] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:40.759117 1342 tablet_bootstrap.cc:654] T bd51a72030254a1ea9d24fde63810a90 P 2b175bd28c5d420ebb309f74c631c031: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:40.760417 1342 log.cc:824] T bd51a72030254a1ea9d24fde63810a90 P 2b175bd28c5d420ebb309f74c631c031: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:40.760414 1344 log.cc:824] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:40.760985 1346 tablet_bootstrap.cc:492] T bd51a72030254a1ea9d24fde63810a90 P 010dcbdf7ba54fa1955ef6e13326129d: Bootstrap starting.
I20260811 18:45:40.763821 1342 tablet_bootstrap.cc:492] T bd51a72030254a1ea9d24fde63810a90 P 2b175bd28c5d420ebb309f74c631c031: No bootstrap required, opened a new log
I20260811 18:45:40.764099 1342 ts_tablet_manager.cc:1397] T bd51a72030254a1ea9d24fde63810a90 P 2b175bd28c5d420ebb309f74c631c031: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260811 18:45:40.764241 1344 tablet_bootstrap.cc:492] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e: No bootstrap required, opened a new log
I20260811 18:45:40.764508 1344 ts_tablet_manager.cc:1397] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260811 18:45:40.764883 1346 tablet_bootstrap.cc:654] T bd51a72030254a1ea9d24fde63810a90 P 010dcbdf7ba54fa1955ef6e13326129d: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:40.766251 1346 log.cc:824] T bd51a72030254a1ea9d24fde63810a90 P 010dcbdf7ba54fa1955ef6e13326129d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:40.770021 1346 tablet_bootstrap.cc:492] T bd51a72030254a1ea9d24fde63810a90 P 010dcbdf7ba54fa1955ef6e13326129d: No bootstrap required, opened a new log
I20260811 18:45:40.770315 1346 ts_tablet_manager.cc:1397] T bd51a72030254a1ea9d24fde63810a90 P 010dcbdf7ba54fa1955ef6e13326129d: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 18:45:40.772295 1342 raft_consensus.cc:348] T bd51a72030254a1ea9d24fde63810a90 P 2b175bd28c5d420ebb309f74c631c031 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010dcbdf7ba54fa1955ef6e13326129d" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 43877 } } peers { permanent_uuid: "f25a656a146f4edd961abc7c98b3b88e" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 33269 } } peers { permanent_uuid: "2b175bd28c5d420ebb309f74c631c031" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 43895 } }
I20260811 18:45:40.772665 1342 raft_consensus.cc:374] T bd51a72030254a1ea9d24fde63810a90 P 2b175bd28c5d420ebb309f74c631c031 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:40.772789 1342 raft_consensus.cc:729] T bd51a72030254a1ea9d24fde63810a90 P 2b175bd28c5d420ebb309f74c631c031 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b175bd28c5d420ebb309f74c631c031, State: Initialized, Role: FOLLOWER
I20260811 18:45:40.773447 1342 consensus_queue.cc:260] T bd51a72030254a1ea9d24fde63810a90 P 2b175bd28c5d420ebb309f74c631c031 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010dcbdf7ba54fa1955ef6e13326129d" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 43877 } } peers { permanent_uuid: "f25a656a146f4edd961abc7c98b3b88e" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 33269 } } peers { permanent_uuid: "2b175bd28c5d420ebb309f74c631c031" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 43895 } }
I20260811 18:45:40.774518 1314 heartbeater.cc:503] Master 127.30.185.190:38489 was elected leader, sending a full tablet report...
I20260811 18:45:40.775234 1342 ts_tablet_manager.cc:1428] T bd51a72030254a1ea9d24fde63810a90 P 2b175bd28c5d420ebb309f74c631c031: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260811 18:45:40.775086 1344 raft_consensus.cc:348] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010dcbdf7ba54fa1955ef6e13326129d" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 43877 } } peers { permanent_uuid: "f25a656a146f4edd961abc7c98b3b88e" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 33269 } } peers { permanent_uuid: "2b175bd28c5d420ebb309f74c631c031" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 43895 } }
I20260811 18:45:40.775492 1344 raft_consensus.cc:374] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:40.775589 1344 raft_consensus.cc:729] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f25a656a146f4edd961abc7c98b3b88e, State: Initialized, Role: FOLLOWER
I20260811 18:45:40.776185 1344 consensus_queue.cc:260] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010dcbdf7ba54fa1955ef6e13326129d" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 43877 } } peers { permanent_uuid: "f25a656a146f4edd961abc7c98b3b88e" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 33269 } } peers { permanent_uuid: "2b175bd28c5d420ebb309f74c631c031" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 43895 } }
I20260811 18:45:40.777927 1344 ts_tablet_manager.cc:1428] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260811 18:45:40.780995 1360 raft_consensus.cc:482] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:45:40.781037 1346 raft_consensus.cc:348] T bd51a72030254a1ea9d24fde63810a90 P 010dcbdf7ba54fa1955ef6e13326129d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010dcbdf7ba54fa1955ef6e13326129d" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 43877 } } peers { permanent_uuid: "f25a656a146f4edd961abc7c98b3b88e" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 33269 } } peers { permanent_uuid: "2b175bd28c5d420ebb309f74c631c031" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 43895 } }
I20260811 18:45:40.781550 1346 raft_consensus.cc:374] T bd51a72030254a1ea9d24fde63810a90 P 010dcbdf7ba54fa1955ef6e13326129d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:40.781334 1360 raft_consensus.cc:504] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010dcbdf7ba54fa1955ef6e13326129d" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 43877 } } peers { permanent_uuid: "f25a656a146f4edd961abc7c98b3b88e" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 33269 } } peers { permanent_uuid: "2b175bd28c5d420ebb309f74c631c031" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 43895 } }
I20260811 18:45:40.781680 1346 raft_consensus.cc:729] T bd51a72030254a1ea9d24fde63810a90 P 010dcbdf7ba54fa1955ef6e13326129d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 010dcbdf7ba54fa1955ef6e13326129d, State: Initialized, Role: FOLLOWER
I20260811 18:45:40.782317 1346 consensus_queue.cc:260] T bd51a72030254a1ea9d24fde63810a90 P 010dcbdf7ba54fa1955ef6e13326129d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010dcbdf7ba54fa1955ef6e13326129d" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 43877 } } peers { permanent_uuid: "f25a656a146f4edd961abc7c98b3b88e" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 33269 } } peers { permanent_uuid: "2b175bd28c5d420ebb309f74c631c031" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 43895 } }
I20260811 18:45:40.783761 1086 heartbeater.cc:503] Master 127.30.185.190:38489 was elected leader, sending a full tablet report...
I20260811 18:45:40.784349 1346 ts_tablet_manager.cc:1428] T bd51a72030254a1ea9d24fde63810a90 P 010dcbdf7ba54fa1955ef6e13326129d: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260811 18:45:40.786090 1360 leader_election.cc:288] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 010dcbdf7ba54fa1955ef6e13326129d (127.30.185.130:43877), 2b175bd28c5d420ebb309f74c631c031 (127.30.185.131:43895)
W20260811 18:45:40.790364 872 tablet.cc:2369] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:45:40.791916 1333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:40.784527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:37139 (local address 127.30.185.130:43877)
0811 18:45:40.784736 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:40.784743 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:40.785347 (+ 604us) server_negotiation.cc:407] Connection header received
0811 18:45:40.785602 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:40.785608 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:40.785686 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:40.785779 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:40.786768 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.787903 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.790124 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.790994 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.791173 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:40.791251 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:40.791553 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 18:45:40.791675 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 18:45:40.791990 1362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:40.783897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:43877 (local address 127.30.185.129:37139)
0811 18:45:40.785236 (+ 1339us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:40.785256 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:40.785462 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:40.785968 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:40.785973 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:40.786006 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:40.786604 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:40.786618 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.788113 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.788119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:40.789943 (+ 1824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:40.789958 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.791216 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.791223 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:40.791389 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:40.791413 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:45:40.791518 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 18:45:40.791641 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1145,"thread_start_us":928,"threads_started":1}
I20260811 18:45:40.793011 1022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd51a72030254a1ea9d24fde63810a90" candidate_uuid: "f25a656a146f4edd961abc7c98b3b88e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010dcbdf7ba54fa1955ef6e13326129d" is_pre_election: true
I20260811 18:45:40.793227 1363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:40.783903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:43895 (local address 127.30.185.129:51985)
0811 18:45:40.786122 (+ 2219us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:40.786150 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:40.786340 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:40.786672 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:40.786677 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:40.787370 (+ 693us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:40.787738 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:40.787751 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.789212 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.789217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:40.791179 (+ 1962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:40.791191 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.792610 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.792617 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:40.792739 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:40.792761 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 18:45:40.792837 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 18:45:40.792919 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2067,"mutex_wait_us":25,"thread_start_us":1822,"threads_started":1}
I20260811 18:45:40.793344 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:40.785300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:51985 (local address 127.30.185.131:43895)
0811 18:45:40.785481 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:40.785486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:40.786234 (+ 748us) server_negotiation.cc:407] Connection header received
0811 18:45:40.786470 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:40.786478 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:40.786563 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:40.786654 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:40.787951 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.789007 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.791338 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.792215 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.792387 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:40.792463 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:40.792905 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 18:45:40.793047 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:45:40.793560 1022 raft_consensus.cc:2454] T bd51a72030254a1ea9d24fde63810a90 P 010dcbdf7ba54fa1955ef6e13326129d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f25a656a146f4edd961abc7c98b3b88e in term 0.
I20260811 18:45:40.794207 689 leader_election.cc:302] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 010dcbdf7ba54fa1955ef6e13326129d, f25a656a146f4edd961abc7c98b3b88e; no voters:
I20260811 18:45:40.794231 1243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd51a72030254a1ea9d24fde63810a90" candidate_uuid: "f25a656a146f4edd961abc7c98b3b88e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b175bd28c5d420ebb309f74c631c031" is_pre_election: true
I20260811 18:45:40.794643 1360 raft_consensus.cc:2781] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:45:40.794718 1243 raft_consensus.cc:2454] T bd51a72030254a1ea9d24fde63810a90 P 2b175bd28c5d420ebb309f74c631c031 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f25a656a146f4edd961abc7c98b3b88e in term 0.
I20260811 18:45:40.794803 1360 raft_consensus.cc:482] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:45:40.794914 1360 raft_consensus.cc:3037] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:40.797708 1360 raft_consensus.cc:504] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010dcbdf7ba54fa1955ef6e13326129d" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 43877 } } peers { permanent_uuid: "f25a656a146f4edd961abc7c98b3b88e" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 33269 } } peers { permanent_uuid: "2b175bd28c5d420ebb309f74c631c031" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 43895 } }
I20260811 18:45:40.798213 1360 leader_election.cc:288] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e [CANDIDATE]: Term 1 election: Requested vote from peers 010dcbdf7ba54fa1955ef6e13326129d (127.30.185.130:43877), 2b175bd28c5d420ebb309f74c631c031 (127.30.185.131:43895)
I20260811 18:45:40.798658 1022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd51a72030254a1ea9d24fde63810a90" candidate_uuid: "f25a656a146f4edd961abc7c98b3b88e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "010dcbdf7ba54fa1955ef6e13326129d"
I20260811 18:45:40.798678 1243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd51a72030254a1ea9d24fde63810a90" candidate_uuid: "f25a656a146f4edd961abc7c98b3b88e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b175bd28c5d420ebb309f74c631c031"
I20260811 18:45:40.798840 1022 raft_consensus.cc:3037] T bd51a72030254a1ea9d24fde63810a90 P 010dcbdf7ba54fa1955ef6e13326129d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:40.798882 1243 raft_consensus.cc:3037] T bd51a72030254a1ea9d24fde63810a90 P 2b175bd28c5d420ebb309f74c631c031 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:40.801043 1243 raft_consensus.cc:2454] T bd51a72030254a1ea9d24fde63810a90 P 2b175bd28c5d420ebb309f74c631c031 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f25a656a146f4edd961abc7c98b3b88e in term 1.
I20260811 18:45:40.801048 1022 raft_consensus.cc:2454] T bd51a72030254a1ea9d24fde63810a90 P 010dcbdf7ba54fa1955ef6e13326129d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f25a656a146f4edd961abc7c98b3b88e in term 1.
I20260811 18:45:40.801487 685 leader_election.cc:302] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b175bd28c5d420ebb309f74c631c031, f25a656a146f4edd961abc7c98b3b88e; no voters:
I20260811 18:45:40.801811 1360 raft_consensus.cc:2781] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:45:40.802258 1360 raft_consensus.cc:686] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e [term 1 LEADER]: Becoming Leader. State: Replica: f25a656a146f4edd961abc7c98b3b88e, State: Running, Role: LEADER
I20260811 18:45:40.802719 1360 consensus_queue.cc:237] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010dcbdf7ba54fa1955ef6e13326129d" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 43877 } } peers { permanent_uuid: "f25a656a146f4edd961abc7c98b3b88e" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 33269 } } peers { permanent_uuid: "2b175bd28c5d420ebb309f74c631c031" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 43895 } }
I20260811 18:45:40.807183 548 catalog_manager.cc:5495] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e reported cstate change: term changed from 0 to 1, leader changed from <none> to f25a656a146f4edd961abc7c98b3b88e (127.30.185.129). New cstate: current_term: 1 leader_uuid: "f25a656a146f4edd961abc7c98b3b88e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "010dcbdf7ba54fa1955ef6e13326129d" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 43877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f25a656a146f4edd961abc7c98b3b88e" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 33269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b175bd28c5d420ebb309f74c631c031" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 43895 } health_report { overall_health: UNKNOWN } } }
I20260811 18:45:40.827688 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:40.822008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48792 (local address 127.30.185.129:33269)
0811 18:45:40.822226 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:40.822233 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:40.822342 (+ 109us) server_negotiation.cc:407] Connection header received
0811 18:45:40.822580 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:40.822584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:40.822657 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:40.822832 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:45:40.823224 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.823944 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.825434 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.825847 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.825999 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:40.826619 (+ 620us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:45:40.827093 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:45:40.827275 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:40.827417 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 18:45:40.827465 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 18:45:40.835752 1243 raft_consensus.cc:1261] T bd51a72030254a1ea9d24fde63810a90 P 2b175bd28c5d420ebb309f74c631c031 [term 1 FOLLOWER]: Refusing update from remote peer f25a656a146f4edd961abc7c98b3b88e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:45:40.835822 1022 raft_consensus.cc:1261] T bd51a72030254a1ea9d24fde63810a90 P 010dcbdf7ba54fa1955ef6e13326129d [term 1 FOLLOWER]: Refusing update from remote peer f25a656a146f4edd961abc7c98b3b88e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:45:40.836719 1368 consensus_queue.cc:1035] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b175bd28c5d420ebb309f74c631c031" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 43895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:45:40.837090 1360 consensus_queue.cc:1035] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e [LEADER]: Connected to new peer: Peer: permanent_uuid: "010dcbdf7ba54fa1955ef6e13326129d" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 43877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:45:40.839646 1087 tablet.cc:2369] T bd51a72030254a1ea9d24fde63810a90 P 010dcbdf7ba54fa1955ef6e13326129d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:45:40.856819 1375 mvcc.cc:205] Tried to move back new op lower bound from 7317397261653516288 to 7317397261531418624. Current Snapshot: MvccSnapshot[applied={T|T < 7317397261653516288 or (T in {7317397261653516288})}]
I20260811 18:45:40.872514 547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:45:40.872786 547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:45:40.876716 547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 864c5fa9c2bb43a3844bc51849be666f: Sending DeleteTablet for 3 replicas of tablet bd51a72030254a1ea9d24fde63810a90
I20260811 18:45:40.877902 988 tablet_service.cc:1514] Processing DeleteTablet for tablet bd51a72030254a1ea9d24fde63810a90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:45:40 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:59034
I20260811 18:45:40.878161 764 tablet_service.cc:1514] Processing DeleteTablet for tablet bd51a72030254a1ea9d24fde63810a90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:45:40 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:48782
I20260811 18:45:40.878983 1215 tablet_service.cc:1514] Processing DeleteTablet for tablet bd51a72030254a1ea9d24fde63810a90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:45:40 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:54640
I20260811 18:45:40.879341 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 660
I20260811 18:45:40.890940 1386 tablet_replica.cc:330] T bd51a72030254a1ea9d24fde63810a90 P 010dcbdf7ba54fa1955ef6e13326129d: stopping tablet replica
I20260811 18:45:40.891525 1386 raft_consensus.cc:2229] T bd51a72030254a1ea9d24fde63810a90 P 010dcbdf7ba54fa1955ef6e13326129d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:45:40.892148 1386 raft_consensus.cc:2258] T bd51a72030254a1ea9d24fde63810a90 P 010dcbdf7ba54fa1955ef6e13326129d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:45:40.894912 1386 ts_tablet_manager.cc:1905] T bd51a72030254a1ea9d24fde63810a90 P 010dcbdf7ba54fa1955ef6e13326129d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:45:40.899713 1387 tablet_replica.cc:330] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e: stopping tablet replica
I20260811 18:45:40.900220 1387 raft_consensus.cc:2229] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:45:40.900972 1387 raft_consensus.cc:2258] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:45:40.903450 1386 ts_tablet_manager.cc:1918] T bd51a72030254a1ea9d24fde63810a90 P 010dcbdf7ba54fa1955ef6e13326129d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:45:40.903579 1387 ts_tablet_manager.cc:1905] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:45:40.903641 1386 log.cc:1196] T bd51a72030254a1ea9d24fde63810a90 P 010dcbdf7ba54fa1955ef6e13326129d: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/bd51a72030254a1ea9d24fde63810a90
I20260811 18:45:40.904235 1386 ts_tablet_manager.cc:1939] T bd51a72030254a1ea9d24fde63810a90 P 010dcbdf7ba54fa1955ef6e13326129d: Deleting consensus metadata
I20260811 18:45:40.906315 526 catalog_manager.cc:4841] TS 010dcbdf7ba54fa1955ef6e13326129d (127.30.185.130:43877): tablet bd51a72030254a1ea9d24fde63810a90 (table test-table [id=53dc7308b2e044bbb8cce169d891bc00]) successfully deleted
I20260811 18:45:40.909999 1387 ts_tablet_manager.cc:1918] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:45:40.910260 1387 log.cc:1196] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-0/wal/wals/bd51a72030254a1ea9d24fde63810a90
I20260811 18:45:40.910395 1388 tablet_replica.cc:330] T bd51a72030254a1ea9d24fde63810a90 P 2b175bd28c5d420ebb309f74c631c031: stopping tablet replica
I20260811 18:45:40.910845 1387 ts_tablet_manager.cc:1939] T bd51a72030254a1ea9d24fde63810a90 P f25a656a146f4edd961abc7c98b3b88e: Deleting consensus metadata
I20260811 18:45:40.910897 1388 raft_consensus.cc:2229] T bd51a72030254a1ea9d24fde63810a90 P 2b175bd28c5d420ebb309f74c631c031 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:45:40.911425 1388 raft_consensus.cc:2258] T bd51a72030254a1ea9d24fde63810a90 P 2b175bd28c5d420ebb309f74c631c031 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:45:40.913817 1388 ts_tablet_manager.cc:1905] T bd51a72030254a1ea9d24fde63810a90 P 2b175bd28c5d420ebb309f74c631c031: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:45:40.914275 522 catalog_manager.cc:4841] TS f25a656a146f4edd961abc7c98b3b88e (127.30.185.129:33269): tablet bd51a72030254a1ea9d24fde63810a90 (table test-table [id=53dc7308b2e044bbb8cce169d891bc00]) successfully deleted
I20260811 18:45:40.919200 1388 ts_tablet_manager.cc:1918] T bd51a72030254a1ea9d24fde63810a90 P 2b175bd28c5d420ebb309f74c631c031: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:45:40.919399 1388 log.cc:1196] T bd51a72030254a1ea9d24fde63810a90 P 2b175bd28c5d420ebb309f74c631c031: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/bd51a72030254a1ea9d24fde63810a90
I20260811 18:45:40.919878 1388 ts_tablet_manager.cc:1939] T bd51a72030254a1ea9d24fde63810a90 P 2b175bd28c5d420ebb309f74c631c031: Deleting consensus metadata
Aug 11 18:45:40 dist-test-slave-z8x9 krb5kdc[474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.129@KRBTEST.COM
I20260811 18:45:40.922026 522 catalog_manager.cc:4841] TS 2b175bd28c5d420ebb309f74c631c031 (127.30.185.131:43895): tablet bd51a72030254a1ea9d24fde63810a90 (table test-table [id=53dc7308b2e044bbb8cce169d891bc00]) successfully deleted
I20260811 18:45:40.928668 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:40.881951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48802 (local address 127.30.185.129:33269)
0811 18:45:40.882180 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:40.882189 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:40.909784 (+ 27595us) server_negotiation.cc:407] Connection header received
0811 18:45:40.910084 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:40.910090 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:40.910188 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:40.910274 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:40.912210 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.913455 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.915411 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:40.916108 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:40.916253 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:40.922039 (+ 5786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:40.922052 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:40.922067 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:40.922110 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:40.925571 (+ 3461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:40.926362 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:40.926370 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:40.926372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:40.926545 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:40.926952 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:40.926957 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:40.926967 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:40.927340 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:40.927681 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:40.927971 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 18:45:40.928227 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:45:40.929666 866 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:41.031489 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 660
I20260811 18:45:41.056293 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 880
Aug 11 18:45:41 dist-test-slave-z8x9 krb5kdc[474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.130@KRBTEST.COM
I20260811 18:45:41.072290 1333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:41.057254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59048 (local address 127.30.185.130:43877)
0811 18:45:41.057438 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:41.057442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:41.057480 (+ 38us) server_negotiation.cc:407] Connection header received
0811 18:45:41.057644 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:41.057648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:41.057711 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:41.057792 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:41.059180 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:41.059948 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:41.061424 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:41.061833 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:41.061928 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:41.066492 (+ 4564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:41.066508 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:41.066525 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:41.066566 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:41.069694 (+ 3128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:41.070396 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:41.070404 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:41.070406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:41.070545 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:41.070877 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:41.070881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:41.070882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:41.071239 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:41.071467 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:41.071723 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 18:45:41.071897 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 18:45:41.073051 1077 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:41.163447 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 880
I20260811 18:45:41.191247 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 1098
Aug 11 18:45:41 dist-test-slave-z8x9 krb5kdc[474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.131@KRBTEST.COM
I20260811 18:45:41.207466 1341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:41.192120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54644 (local address 127.30.185.131:43895)
0811 18:45:41.192331 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:41.192339 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:41.192383 (+ 44us) server_negotiation.cc:407] Connection header received
0811 18:45:41.192613 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:41.192617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:41.192684 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:41.192803 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:41.194291 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:41.195040 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:41.196361 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:41.196785 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:41.196930 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:41.201755 (+ 4825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:41.201771 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:41.201787 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:41.201829 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:41.204502 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:41.205259 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:41.205267 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:41.205269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:41.205416 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:41.205820 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:41.205826 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:41.205829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:41.206194 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:41.206539 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:41.206880 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 18:45:41.207111 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 18:45:41.208114 1308 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:41.300319 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 1098
I20260811 18:45:41.322563 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 493
I20260811 18:45:41.323764 606 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:41.436000 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 493
2026-08-11T18:45:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4861 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T18:45:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:45:41Z Disabled control of system clock
I20260811 18:45:41.541296 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:43429
--webserver_interface=127.30.185.190
--webserver_port=0
--builtin_ntp_servers=127.30.185.148:38269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:43429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:45:41.892839 1401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:41.893154 1401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:41.893337 1401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:41.900843 1401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:45:41.900952 1401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:41.901047 1401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:41.901167 1401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:45:41.901278 1401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:45:41.910880 1401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:43429
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:43429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:41.912210 1401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:41.914083 1401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:41.927289 1416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:41.927438 1407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:41.927343 1408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:41.928653 1401 server_base.cc:1034] running on GCE node
I20260811 18:45:41.929621 1401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:41.931095 1401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:41.932325 1401 hybrid_clock.cc:648] HybridClock initialized: now 1786473941932285 us; error 26 us; skew 500 ppm
I20260811 18:45:41.932678 1401 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:45:41.939186 1401 webserver.cc:460] Webserver started at http://127.30.185.190:34603/ using document root <none> and password file <none>
I20260811 18:45:41.939913 1401 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:41.940012 1401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:41.940306 1401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:41.943504 1401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "27373c59d27745b299f91fe567d36db6"
format_stamp: "Formatted at 2026-08-11 18:45:41 on dist-test-slave-z8x9"
server_key: "a1c7bf53b74a510625a084114e414c5a"
server_key_iv: "cc09ae91ba24e41e898033d8cfd1f034"
server_key_version: "encryptionkey@0"
I20260811 18:45:41.944298 1401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "27373c59d27745b299f91fe567d36db6"
format_stamp: "Formatted at 2026-08-11 18:45:41 on dist-test-slave-z8x9"
server_key: "a1c7bf53b74a510625a084114e414c5a"
server_key_iv: "cc09ae91ba24e41e898033d8cfd1f034"
server_key_version: "encryptionkey@0"
I20260811 18:45:41.949664 1401 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 18:45:41.953455 1426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:41.954893 1401 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:45:41.955058 1401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "27373c59d27745b299f91fe567d36db6"
format_stamp: "Formatted at 2026-08-11 18:45:41 on dist-test-slave-z8x9"
server_key: "a1c7bf53b74a510625a084114e414c5a"
server_key_iv: "cc09ae91ba24e41e898033d8cfd1f034"
server_key_version: "encryptionkey@0"
I20260811 18:45:41.955304 1401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:41.970536 1401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:41.971442 1401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:41.971702 1401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:41.991734 1401 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:43429
I20260811 18:45:41.991916 1508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:43429 every 8 connection(s)
I20260811 18:45:41.993534 1401 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:45:41.997398 1509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:42.001992 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 1401
I20260811 18:45:42.002203 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:45:42.002687 31462 external_mini_cluster.cc:1428] Setting key 8bed95799d607b2c0f8aae3b646b6670
I20260811 18:45:42.008553 1509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6: Bootstrap starting.
I20260811 18:45:42.012343 1509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:42.013676 1509 log.cc:824] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:42.017463 1509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6: No bootstrap required, opened a new log
I20260811 18:45:42.017590 1516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:42.004670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42602 (local address 127.30.185.190:43429)
0811 18:45:42.005816 (+ 1146us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:42.005830 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:42.005874 (+ 44us) server_negotiation.cc:407] Connection header received
0811 18:45:42.007619 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:42.007642 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:42.008093 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:42.008431 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:42.009102 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:42.010522 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:42.012573 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:42.013355 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:42.013503 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:42.014925 (+ 1422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:42.014939 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:42.014961 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:42.015691 (+ 730us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:42.015790 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:42.015818 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:42.015930 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:42.016172 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 18:45:42.016564 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":452,"threads_started":1}
I20260811 18:45:42.025616 1509 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27373c59d27745b299f91fe567d36db6" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43429 } }
I20260811 18:45:42.026113 1509 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:42.026238 1509 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27373c59d27745b299f91fe567d36db6, State: Initialized, Role: FOLLOWER
I20260811 18:45:42.026866 1509 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27373c59d27745b299f91fe567d36db6" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43429 } }
I20260811 18:45:42.027122 1509 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:45:42.027278 1509 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:45:42.027478 1509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:42.030088 1509 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27373c59d27745b299f91fe567d36db6" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43429 } }
I20260811 18:45:42.030627 1509 leader_election.cc:302] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27373c59d27745b299f91fe567d36db6; no voters:
I20260811 18:45:42.031232 1509 leader_election.cc:288] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:45:42.031414 1519 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:45:42.032244 1519 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6 [term 1 LEADER]: Becoming Leader. State: Replica: 27373c59d27745b299f91fe567d36db6, State: Running, Role: LEADER
I20260811 18:45:42.032784 1519 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27373c59d27745b299f91fe567d36db6" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43429 } }
I20260811 18:45:42.033766 1509 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:45:42.035910 1527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27373c59d27745b299f91fe567d36db6. Latest consensus state: current_term: 1 leader_uuid: "27373c59d27745b299f91fe567d36db6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27373c59d27745b299f91fe567d36db6" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43429 } } }
I20260811 18:45:42.036394 1527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6 [sys.catalog]: This master's current role is: LEADER
I20260811 18:45:42.036032 1526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27373c59d27745b299f91fe567d36db6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27373c59d27745b299f91fe567d36db6" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43429 } } }
I20260811 18:45:42.036829 1526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6 [sys.catalog]: This master's current role is: LEADER
I20260811 18:45:42.038038 1535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:45:42.043521 1535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:45:42.052428 1535 catalog_manager.cc:1347] Generated new cluster ID: 137cc69ab0164e02800c69e4756c85dc
I20260811 18:45:42.052563 1535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:45:42.075855 1535 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:45:42.077297 1535 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:45:42.082967 1535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6: Generated new TSK 0
I20260811 18:45:42.083824 1535 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:45:42.097875 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:43429
--builtin_ntp_servers=127.30.185.148:38269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:45:42.477849 1557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:42.478142 1557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:42.478304 1557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:42.485569 1557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:42.485675 1557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:42.485905 1557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:45:42.495101 1557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:43429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:42.496479 1557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:42.498308 1557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:42.511844 1570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:42.511931 1571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:42.511886 1573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:42.512997 1557 server_base.cc:1034] running on GCE node
I20260811 18:45:42.513701 1557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:42.514689 1557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:42.515897 1557 hybrid_clock.cc:648] HybridClock initialized: now 1786473942515854 us; error 41 us; skew 500 ppm
I20260811 18:45:42.516227 1557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:45:42.518690 1557 webserver.cc:460] Webserver started at http://127.30.185.129:43239/ using document root <none> and password file <none>
I20260811 18:45:42.519405 1557 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:42.519518 1557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:42.519863 1557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:42.522969 1557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "32130acdc9dd4e82b1afcba7f40682d1"
format_stamp: "Formatted at 2026-08-11 18:45:42 on dist-test-slave-z8x9"
server_key: "a1614d1af1e9219d8239b91bf27a2705"
server_key_iv: "781285e9e3e9101cc6fce68f956c43f9"
server_key_version: "encryptionkey@0"
I20260811 18:45:42.523797 1557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "32130acdc9dd4e82b1afcba7f40682d1"
format_stamp: "Formatted at 2026-08-11 18:45:42 on dist-test-slave-z8x9"
server_key: "a1614d1af1e9219d8239b91bf27a2705"
server_key_iv: "781285e9e3e9101cc6fce68f956c43f9"
server_key_version: "encryptionkey@0"
I20260811 18:45:42.529028 1557 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 18:45:42.532528 1580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:42.533951 1557 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:45:42.534124 1557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "32130acdc9dd4e82b1afcba7f40682d1"
format_stamp: "Formatted at 2026-08-11 18:45:42 on dist-test-slave-z8x9"
server_key: "a1614d1af1e9219d8239b91bf27a2705"
server_key_iv: "781285e9e3e9101cc6fce68f956c43f9"
server_key_version: "encryptionkey@0"
I20260811 18:45:42.534365 1557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:42.568462 1557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:42.569391 1557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:42.569720 1557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:42.570804 1557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:45:42.572359 1557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:45:42.572492 1557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:42.572631 1557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:45:42.572726 1557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:42.607710 1557 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:46683
I20260811 18:45:42.607816 1798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:46683 every 8 connection(s)
I20260811 18:45:42.609400 1557 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:45:42.618662 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 1557
I20260811 18:45:42.618860 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:45:42.619433 31462 external_mini_cluster.cc:1428] Setting key 8b4b6730dbc30bb7a8139331d8500d2f
I20260811 18:45:42.622275 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:43429
--builtin_ntp_servers=127.30.185.148:38269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 18:45:42.622452 1803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:42.612041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:48369 (local address 127.30.185.190:43429)
0811 18:45:42.612477 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:42.612482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:42.613783 (+ 1301us) server_negotiation.cc:407] Connection header received
0811 18:45:42.615499 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:42.615504 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:42.615583 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:42.615682 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:42.616674 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:42.617480 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:42.619576 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:42.619974 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:42.620113 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:42.621600 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:42.621605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:42.621608 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:42.621626 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:42.621666 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:42.621679 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:42.621784 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:42.622008 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 18:45:42.622102 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":227,"threads_started":1}
I20260811 18:45:42.623433 1805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:42.612664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:43429 (local address 127.30.185.129:48369)
0811 18:45:42.613596 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:42.613638 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:42.615253 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:42.615895 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:42.615903 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:42.615981 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:45:42.616530 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:42.616546 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:42.617639 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:42.617644 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:42.619391 (+ 1747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:42.619407 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:42.620208 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:42.620216 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:42.620456 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:42.621253 (+ 797us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:45:42.621256 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:45:42.621324 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:45:42.621399 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:45:42.621779 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:45:42.621791 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:45:42.621802 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 18:45:42.621991 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 18:45:42.622377 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":273,"threads_started":1}
I20260811 18:45:42.624961 1799 heartbeater.cc:347] Connected to a master server at 127.30.185.190:43429
I20260811 18:45:42.625312 1799 heartbeater.cc:464] Registering TS with master...
I20260811 18:45:42.626235 1799 heartbeater.cc:511] Master 127.30.185.190:43429 requested a full tablet report, sending...
I20260811 18:45:42.628585 1451 ts_manager.cc:194] Registered new tserver with Master: 32130acdc9dd4e82b1afcba7f40682d1 (127.30.185.129:46683)
I20260811 18:45:42.630744 1451 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.185.129:48369
W20260811 18:45:42.975991 1806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:42.976354 1806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:42.976522 1806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:42.984036 1806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:42.984230 1806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:42.984416 1806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:45:42.994230 1806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:43429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:42.995486 1806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:42.997359 1806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:43.010901 1816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:43.010886 1822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:43.010900 1817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:43.011564 1806 server_base.cc:1034] running on GCE node
I20260811 18:45:43.012205 1806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:43.013239 1806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:43.014482 1806 hybrid_clock.cc:648] HybridClock initialized: now 1786473943014430 us; error 46 us; skew 500 ppm
I20260811 18:45:43.014814 1806 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:45:43.017383 1806 webserver.cc:460] Webserver started at http://127.30.185.130:40491/ using document root <none> and password file <none>
I20260811 18:45:43.018106 1806 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:43.018247 1806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:43.018584 1806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:43.021564 1806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "8c61d200e5c3498baaf9c99081547b0a"
format_stamp: "Formatted at 2026-08-11 18:45:43 on dist-test-slave-z8x9"
server_key: "e852c38782f501c69611c3388bc730cd"
server_key_iv: "f186ea41b27f12e00e5f75068641444f"
server_key_version: "encryptionkey@0"
I20260811 18:45:43.022392 1806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "8c61d200e5c3498baaf9c99081547b0a"
format_stamp: "Formatted at 2026-08-11 18:45:43 on dist-test-slave-z8x9"
server_key: "e852c38782f501c69611c3388bc730cd"
server_key_iv: "f186ea41b27f12e00e5f75068641444f"
server_key_version: "encryptionkey@0"
I20260811 18:45:43.027603 1806 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:45:43.031186 1835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:43.032575 1806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:45:43.032760 1806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "8c61d200e5c3498baaf9c99081547b0a"
format_stamp: "Formatted at 2026-08-11 18:45:43 on dist-test-slave-z8x9"
server_key: "e852c38782f501c69611c3388bc730cd"
server_key_iv: "f186ea41b27f12e00e5f75068641444f"
server_key_version: "encryptionkey@0"
I20260811 18:45:43.032989 1806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:43.055328 1806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:43.056185 1806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:43.056520 1806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:43.057657 1806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:45:43.059295 1806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:45:43.059429 1806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:43.059564 1806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:45:43.059656 1806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:43.092608 1806 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:42433
I20260811 18:45:43.092732 2046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:42433 every 8 connection(s)
I20260811 18:45:43.094051 1806 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:45:43.098325 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 1806
I20260811 18:45:43.098551 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:45:43.099036 31462 external_mini_cluster.cc:1428] Setting key c278e9ada8df2becbc3be912a1ed1ae7
I20260811 18:45:43.102172 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:43429
--builtin_ntp_servers=127.30.185.148:38269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 18:45:43.107558 1803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:43.096692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:41587 (local address 127.30.185.190:43429)
0811 18:45:43.096884 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:43.096889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:43.098318 (+ 1429us) server_negotiation.cc:407] Connection header received
0811 18:45:43.100444 (+ 2126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:43.100451 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:43.100546 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:43.100664 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:43.101943 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:43.102765 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:43.104372 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:43.104778 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:43.104921 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:43.106703 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:43.106707 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:43.106711 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:43.106728 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:43.106759 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:43.106771 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:43.106887 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:43.107131 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 18:45:43.107232 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 18:45:43.108610 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:43.097145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:43429 (local address 127.30.185.130:41587)
0811 18:45:43.098133 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:43.098170 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:43.100150 (+ 1980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:43.100954 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:43.100966 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:43.101085 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:45:43.101761 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:43.101779 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:43.102936 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:43.102949 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:43.104202 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:43.104214 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:43.105027 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:43.105035 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:43.105347 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:43.106104 (+ 757us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:45:43.106107 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:45:43.106148 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:45:43.106233 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:45:43.106889 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:45:43.106903 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:45:43.106914 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 18:45:43.107100 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 18:45:43.107525 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":275,"threads_started":1}
I20260811 18:45:43.110240 2049 heartbeater.cc:347] Connected to a master server at 127.30.185.190:43429
I20260811 18:45:43.110558 2049 heartbeater.cc:464] Registering TS with master...
I20260811 18:45:43.111375 2049 heartbeater.cc:511] Master 127.30.185.190:43429 requested a full tablet report, sending...
I20260811 18:45:43.112947 1451 ts_manager.cc:194] Registered new tserver with Master: 8c61d200e5c3498baaf9c99081547b0a (127.30.185.130:42433)
I20260811 18:45:43.114178 1451 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.185.130:41587
W20260811 18:45:43.449438 2057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:43.449733 2057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:43.449908 2057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:43.457436 2057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:43.457549 2057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:43.457729 2057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:45:43.467182 2057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:43429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:43.468569 2057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:43.470500 2057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:43.484176 2072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:43.484118 2067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:43.484606 2068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:43.484850 2057 server_base.cc:1034] running on GCE node
I20260811 18:45:43.485608 2057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:43.486508 2057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:43.487710 2057 hybrid_clock.cc:648] HybridClock initialized: now 1786473943487669 us; error 36 us; skew 500 ppm
I20260811 18:45:43.488039 2057 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:45:43.490373 2057 webserver.cc:460] Webserver started at http://127.30.185.131:46397/ using document root <none> and password file <none>
I20260811 18:45:43.491092 2057 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:43.491218 2057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:43.491560 2057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:43.494585 2057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "02eb86b727b9472db54078654d5742bd"
format_stamp: "Formatted at 2026-08-11 18:45:43 on dist-test-slave-z8x9"
server_key: "6e571001b1940c5794c2c5e6de4042c1"
server_key_iv: "f461109fad9d004ff0758458922f5283"
server_key_version: "encryptionkey@0"
I20260811 18:45:43.495375 2057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "02eb86b727b9472db54078654d5742bd"
format_stamp: "Formatted at 2026-08-11 18:45:43 on dist-test-slave-z8x9"
server_key: "6e571001b1940c5794c2c5e6de4042c1"
server_key_iv: "f461109fad9d004ff0758458922f5283"
server_key_version: "encryptionkey@0"
I20260811 18:45:43.500341 2057 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 18:45:43.503727 2083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:43.505057 2057 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:45:43.505275 2057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "02eb86b727b9472db54078654d5742bd"
format_stamp: "Formatted at 2026-08-11 18:45:43 on dist-test-slave-z8x9"
server_key: "6e571001b1940c5794c2c5e6de4042c1"
server_key_iv: "f461109fad9d004ff0758458922f5283"
server_key_version: "encryptionkey@0"
I20260811 18:45:43.505487 2057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:43.549609 2057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:43.550529 2057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:43.550858 2057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:43.551901 2057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:45:43.553447 2057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:45:43.553537 2057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:43.553715 2057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:45:43.553815 2057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:43.588680 2057 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:42915
I20260811 18:45:43.588732 2284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:42915 every 8 connection(s)
I20260811 18:45:43.590059 2057 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:45:43.597366 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 2057
I20260811 18:45:43.597579 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:45:43.598183 31462 external_mini_cluster.cc:1428] Setting key 447d3a2b9bbe267dbee8efccf46a68eb
I20260811 18:45:43.603560 1803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:43.592730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:54741 (local address 127.30.185.190:43429)
0811 18:45:43.592955 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:43.592962 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:43.594580 (+ 1618us) server_negotiation.cc:407] Connection header received
0811 18:45:43.596407 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:43.596412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:43.596481 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:43.596572 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:43.597993 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:43.598948 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:43.600972 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:43.601496 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:43.601592 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:43.602911 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:43.602916 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:43.602919 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:43.602935 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:43.602965 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:43.602977 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:43.603047 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:43.603240 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 18:45:43.603323 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 18:45:43.604581 2288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:43.593404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:43429 (local address 127.30.185.131:54741)
0811 18:45:43.594395 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:43.594446 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:43.596169 (+ 1723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:43.596790 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:43.596798 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:43.596923 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:45:43.597798 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:43.597824 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:43.599107 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:43.599115 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:43.600822 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:43.600838 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:43.601724 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:43.601730 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:43.601925 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:43.602595 (+ 670us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:45:43.602597 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:45:43.602636 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:45:43.602734 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:45:43.603059 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:45:43.603066 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:45:43.603071 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:45:43.603211 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 18:45:43.603487 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":315,"threads_started":1}
I20260811 18:45:43.606056 2285 heartbeater.cc:347] Connected to a master server at 127.30.185.190:43429
I20260811 18:45:43.606421 2285 heartbeater.cc:464] Registering TS with master...
I20260811 18:45:43.607287 2285 heartbeater.cc:511] Master 127.30.185.190:43429 requested a full tablet report, sending...
I20260811 18:45:43.608806 1451 ts_manager.cc:194] Registered new tserver with Master: 02eb86b727b9472db54078654d5742bd (127.30.185.131:42915)
I20260811 18:45:43.609915 1451 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.185.131:54741
I20260811 18:45:43.613449 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:45:43.623819 1803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:43.617626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37002 (local address 127.30.185.190:43429)
0811 18:45:43.617852 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:43.617861 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:43.618282 (+ 421us) server_negotiation.cc:407] Connection header received
0811 18:45:43.618483 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:43.618488 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:43.618547 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:43.618654 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:43.619156 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:43.619847 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:43.621311 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:43.621724 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:43.621869 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:43.623072 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:43.623077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:43.623080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:43.623097 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:43.623127 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:43.623139 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:43.623289 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:43.623494 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 18:45:43.623548 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 18:45:43.627874 1451 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:45:43.634447 1799 heartbeater.cc:503] Master 127.30.185.190:43429 was elected leader, sending a full tablet report...
W20260811 18:45:43.634853 1451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:45:43.665817 2306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:43.649320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56796 (local address 127.30.185.131:42915)
0811 18:45:43.650164 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:43.650176 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:43.650703 (+ 527us) server_negotiation.cc:407] Connection header received
0811 18:45:43.651055 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:43.651068 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:43.651402 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:43.651838 (+ 436us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:43.653051 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:43.655812 (+ 2761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:43.659498 (+ 3686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:43.663152 (+ 3654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:43.663369 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:43.663676 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:43.665189 (+ 1513us) server_negotiation.cc:299] Negotiation successful
0811 18:45:43.665374 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"spinlock_wait_cycles":20352,"thread_start_us":580,"threads_started":1}
I20260811 18:45:43.667004 2307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:43.648771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:42915 (local address 127.0.0.1:56796)
0811 18:45:43.650595 (+ 1824us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:43.650612 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:43.650835 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:43.652199 (+ 1364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:43.652207 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:43.652243 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:43.652710 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:43.652724 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:43.656000 (+ 3276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:43.656005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:43.659212 (+ 3207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:43.659234 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:43.663932 (+ 4698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:43.663940 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:43.664845 (+ 905us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:43.664915 (+ 70us) client_negotiation.cc:769] Sending connection context
0811 18:45:43.666366 (+ 1451us) client_negotiation.cc:240] Negotiation successful
0811 18:45:43.666597 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1702,"mutex_wait_us":1505,"thread_start_us":614,"threads_started":1}
I20260811 18:45:43.667444 2302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:43.648160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:42433 (local address 127.0.0.1:40966)
0811 18:45:43.649109 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:43.649158 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:43.649373 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:43.657827 (+ 8454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:43.657832 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:43.657859 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:43.658405 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:43.658423 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:43.661222 (+ 2799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:43.661230 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:43.664316 (+ 3086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:43.664339 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:43.666366 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:43.666373 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:43.666573 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:43.666693 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 18:45:43.666800 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 18:45:43.666967 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"mutex_wait_us":55,"thread_start_us":1377,"threads_started":1}
I20260811 18:45:43.667560 2303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:43.648424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40966 (local address 127.30.185.130:42433)
0811 18:45:43.657053 (+ 8629us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:43.657063 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:43.657102 (+ 39us) server_negotiation.cc:407] Connection header received
0811 18:45:43.657295 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:43.657304 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:43.657604 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:43.657845 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:43.658670 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:43.660983 (+ 2313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:43.664615 (+ 3632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:43.666113 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:43.666685 (+ 572us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:43.666902 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:43.666980 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 18:45:43.667167 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8477,"thread_start_us":8372,"threads_started":1}
I20260811 18:45:43.671090 1938 tablet_service.cc:1467] Processing CreateTablet for tablet f89dc2a9a0e642b4b40ff3dba189c9c6 (DEFAULT_TABLE table=test-table [id=fd2d7ab3a622437a9240f79488ba68a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:45:43.671308 2166 tablet_service.cc:1467] Processing CreateTablet for tablet f89dc2a9a0e642b4b40ff3dba189c9c6 (DEFAULT_TABLE table=test-table [id=fd2d7ab3a622437a9240f79488ba68a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:45:43.672709 1938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f89dc2a9a0e642b4b40ff3dba189c9c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:43.672803 2166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f89dc2a9a0e642b4b40ff3dba189c9c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:43.674595 2309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:43.650354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:46683 (local address 127.0.0.1:40342)
0811 18:45:43.665237 (+ 14883us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:43.665258 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:43.665384 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:43.666387 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:43.666390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:43.666410 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:43.666798 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:43.666811 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:43.669267 (+ 2456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:43.669272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:43.671332 (+ 2060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:43.671351 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:43.673725 (+ 2374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:43.673732 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:43.673920 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:43.673956 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 18:45:43.674063 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 18:45:43.674216 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14721,"mutex_wait_us":2,"thread_start_us":14567,"threads_started":1}
I20260811 18:45:43.674710 2315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:43.651013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40342 (local address 127.30.185.129:46683)
0811 18:45:43.664875 (+ 13862us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:43.664887 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:43.665515 (+ 628us) server_negotiation.cc:407] Connection header received
0811 18:45:43.665621 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:43.665631 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:43.665928 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:43.666131 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:43.667006 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:43.668451 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:43.671585 (+ 3134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:43.673505 (+ 1920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:43.673718 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:43.673941 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:43.674128 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 18:45:43.674320 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13597,"thread_start_us":13402,"threads_started":1}
I20260811 18:45:43.679049 1671 tablet_service.cc:1467] Processing CreateTablet for tablet f89dc2a9a0e642b4b40ff3dba189c9c6 (DEFAULT_TABLE table=test-table [id=fd2d7ab3a622437a9240f79488ba68a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:45:43.681010 1671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f89dc2a9a0e642b4b40ff3dba189c9c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:43.685338 2316 tablet_bootstrap.cc:492] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 8c61d200e5c3498baaf9c99081547b0a: Bootstrap starting.
I20260811 18:45:43.687153 2317 tablet_bootstrap.cc:492] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 02eb86b727b9472db54078654d5742bd: Bootstrap starting.
I20260811 18:45:43.689976 2316 tablet_bootstrap.cc:654] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 8c61d200e5c3498baaf9c99081547b0a: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:43.690805 2317 tablet_bootstrap.cc:654] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 02eb86b727b9472db54078654d5742bd: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:43.691354 2316 log.cc:824] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 8c61d200e5c3498baaf9c99081547b0a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:43.692107 2317 log.cc:824] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 02eb86b727b9472db54078654d5742bd: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:43.696852 2316 tablet_bootstrap.cc:492] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 8c61d200e5c3498baaf9c99081547b0a: No bootstrap required, opened a new log
I20260811 18:45:43.697297 2316 ts_tablet_manager.cc:1397] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 8c61d200e5c3498baaf9c99081547b0a: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20260811 18:45:43.698288 2321 tablet_bootstrap.cc:492] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1: Bootstrap starting.
I20260811 18:45:43.702945 2321 tablet_bootstrap.cc:654] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:43.705591 2321 log.cc:824] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:43.705986 2317 tablet_bootstrap.cc:492] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 02eb86b727b9472db54078654d5742bd: No bootstrap required, opened a new log
I20260811 18:45:43.706362 2317 ts_tablet_manager.cc:1397] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 02eb86b727b9472db54078654d5742bd: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.008s
I20260811 18:45:43.710117 2316 raft_consensus.cc:348] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 8c61d200e5c3498baaf9c99081547b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61d200e5c3498baaf9c99081547b0a" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 42433 } } peers { permanent_uuid: "32130acdc9dd4e82b1afcba7f40682d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 46683 } } peers { permanent_uuid: "02eb86b727b9472db54078654d5742bd" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42915 } }
I20260811 18:45:43.710597 2316 raft_consensus.cc:374] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 8c61d200e5c3498baaf9c99081547b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:43.710743 2316 raft_consensus.cc:729] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 8c61d200e5c3498baaf9c99081547b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c61d200e5c3498baaf9c99081547b0a, State: Initialized, Role: FOLLOWER
I20260811 18:45:43.710966 2321 tablet_bootstrap.cc:492] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1: No bootstrap required, opened a new log
I20260811 18:45:43.711251 2321 ts_tablet_manager.cc:1397] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.004s
I20260811 18:45:43.711388 2316 consensus_queue.cc:260] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 8c61d200e5c3498baaf9c99081547b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61d200e5c3498baaf9c99081547b0a" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 42433 } } peers { permanent_uuid: "32130acdc9dd4e82b1afcba7f40682d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 46683 } } peers { permanent_uuid: "02eb86b727b9472db54078654d5742bd" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42915 } }
I20260811 18:45:43.719179 2317 raft_consensus.cc:348] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 02eb86b727b9472db54078654d5742bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61d200e5c3498baaf9c99081547b0a" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 42433 } } peers { permanent_uuid: "32130acdc9dd4e82b1afcba7f40682d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 46683 } } peers { permanent_uuid: "02eb86b727b9472db54078654d5742bd" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42915 } }
I20260811 18:45:43.719782 2317 raft_consensus.cc:374] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 02eb86b727b9472db54078654d5742bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:43.719919 2317 raft_consensus.cc:729] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 02eb86b727b9472db54078654d5742bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02eb86b727b9472db54078654d5742bd, State: Initialized, Role: FOLLOWER
I20260811 18:45:43.720820 2317 consensus_queue.cc:260] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 02eb86b727b9472db54078654d5742bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61d200e5c3498baaf9c99081547b0a" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 42433 } } peers { permanent_uuid: "32130acdc9dd4e82b1afcba7f40682d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 46683 } } peers { permanent_uuid: "02eb86b727b9472db54078654d5742bd" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42915 } }
I20260811 18:45:43.722169 2049 heartbeater.cc:503] Master 127.30.185.190:43429 was elected leader, sending a full tablet report...
I20260811 18:45:43.722177 2285 heartbeater.cc:503] Master 127.30.185.190:43429 was elected leader, sending a full tablet report...
I20260811 18:45:43.722548 2321 raft_consensus.cc:348] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61d200e5c3498baaf9c99081547b0a" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 42433 } } peers { permanent_uuid: "32130acdc9dd4e82b1afcba7f40682d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 46683 } } peers { permanent_uuid: "02eb86b727b9472db54078654d5742bd" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42915 } }
I20260811 18:45:43.722951 2316 ts_tablet_manager.cc:1428] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 8c61d200e5c3498baaf9c99081547b0a: Time spent starting tablet: real 0.025s user 0.016s sys 0.007s
I20260811 18:45:43.722990 2321 raft_consensus.cc:374] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:43.723084 2321 raft_consensus.cc:729] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32130acdc9dd4e82b1afcba7f40682d1, State: Initialized, Role: FOLLOWER
I20260811 18:45:43.724145 2321 consensus_queue.cc:260] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61d200e5c3498baaf9c99081547b0a" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 42433 } } peers { permanent_uuid: "32130acdc9dd4e82b1afcba7f40682d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 46683 } } peers { permanent_uuid: "02eb86b727b9472db54078654d5742bd" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42915 } }
I20260811 18:45:43.725019 2317 ts_tablet_manager.cc:1428] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 02eb86b727b9472db54078654d5742bd: Time spent starting tablet: real 0.018s user 0.012s sys 0.003s
I20260811 18:45:43.735028 2321 ts_tablet_manager.cc:1428] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1: Time spent starting tablet: real 0.024s user 0.017s sys 0.007s
W20260811 18:45:43.841941 2287 tablet.cc:2369] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 02eb86b727b9472db54078654d5742bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:45:43.846182 2052 tablet.cc:2369] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 8c61d200e5c3498baaf9c99081547b0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:45:43.861970 1800 tablet.cc:2369] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:45:44.054809 2328 raft_consensus.cc:482] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:45:44.055204 2328 raft_consensus.cc:504] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61d200e5c3498baaf9c99081547b0a" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 42433 } } peers { permanent_uuid: "32130acdc9dd4e82b1afcba7f40682d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 46683 } } peers { permanent_uuid: "02eb86b727b9472db54078654d5742bd" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42915 } }
I20260811 18:45:44.056998 2328 leader_election.cc:288] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c61d200e5c3498baaf9c99081547b0a (127.30.185.130:42433), 02eb86b727b9472db54078654d5742bd (127.30.185.131:42915)
I20260811 18:45:44.064569 2303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:44.057366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:60271 (local address 127.30.185.130:42433)
0811 18:45:44.057582 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:44.057592 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:44.058369 (+ 777us) server_negotiation.cc:407] Connection header received
0811 18:45:44.058657 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:44.058666 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:44.058757 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:44.058856 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:44.059608 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:44.060430 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:44.062726 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:44.063523 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:44.063642 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:44.063694 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:44.064168 (+ 474us) server_negotiation.cc:299] Negotiation successful
0811 18:45:44.064297 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:45:44.064610 2306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:44.057679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:34405 (local address 127.30.185.131:42915)
0811 18:45:44.057902 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:44.057912 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:44.058729 (+ 817us) server_negotiation.cc:407] Connection header received
0811 18:45:44.058904 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:44.058910 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:44.058998 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:44.059185 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:44.059938 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:44.060872 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:44.062824 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:44.063715 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:44.063910 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:44.063982 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:44.064217 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 18:45:44.064364 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 18:45:44.064752 2330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:44.057316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:42433 (local address 127.30.185.129:60271)
0811 18:45:44.058209 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:44.058232 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:44.058449 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:44.058908 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:44.058914 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:44.058956 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:44.059435 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:44.059452 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:44.060594 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:44.060600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:44.062512 (+ 1912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:44.062526 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:44.063753 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:44.063763 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:44.063969 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:44.064011 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 18:45:44.064106 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 18:45:44.064272 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"mutex_wait_us":22,"thread_start_us":525,"threads_started":1}
I20260811 18:45:44.065080 2331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:44.057510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:42915 (local address 127.30.185.129:34405)
0811 18:45:44.058587 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:44.058604 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:44.058735 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:44.059226 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:44.059231 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:44.059266 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:44.059744 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:44.059757 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:44.061018 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:44.061022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:44.062599 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:44.062612 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:44.063910 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:44.063914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:44.064024 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:44.064046 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 18:45:44.064159 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 18:45:44.064293 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":906,"mutex_wait_us":44,"thread_start_us":678,"threads_started":1}
I20260811 18:45:44.065766 1972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f89dc2a9a0e642b4b40ff3dba189c9c6" candidate_uuid: "32130acdc9dd4e82b1afcba7f40682d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61d200e5c3498baaf9c99081547b0a" is_pre_election: true
I20260811 18:45:44.066083 2198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f89dc2a9a0e642b4b40ff3dba189c9c6" candidate_uuid: "32130acdc9dd4e82b1afcba7f40682d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02eb86b727b9472db54078654d5742bd" is_pre_election: true
I20260811 18:45:44.066303 1972 raft_consensus.cc:2454] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 8c61d200e5c3498baaf9c99081547b0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32130acdc9dd4e82b1afcba7f40682d1 in term 0.
I20260811 18:45:44.066558 2198 raft_consensus.cc:2454] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 02eb86b727b9472db54078654d5742bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32130acdc9dd4e82b1afcba7f40682d1 in term 0.
I20260811 18:45:44.067018 1587 leader_election.cc:302] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32130acdc9dd4e82b1afcba7f40682d1, 8c61d200e5c3498baaf9c99081547b0a; no voters:
I20260811 18:45:44.067430 2328 raft_consensus.cc:2781] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:45:44.067587 2328 raft_consensus.cc:482] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:45:44.067698 2328 raft_consensus.cc:3037] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:44.069917 2328 raft_consensus.cc:504] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61d200e5c3498baaf9c99081547b0a" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 42433 } } peers { permanent_uuid: "32130acdc9dd4e82b1afcba7f40682d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 46683 } } peers { permanent_uuid: "02eb86b727b9472db54078654d5742bd" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42915 } }
I20260811 18:45:44.070550 2328 leader_election.cc:288] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1 [CANDIDATE]: Term 1 election: Requested vote from peers 8c61d200e5c3498baaf9c99081547b0a (127.30.185.130:42433), 02eb86b727b9472db54078654d5742bd (127.30.185.131:42915)
I20260811 18:45:44.070887 1972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f89dc2a9a0e642b4b40ff3dba189c9c6" candidate_uuid: "32130acdc9dd4e82b1afcba7f40682d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61d200e5c3498baaf9c99081547b0a"
I20260811 18:45:44.071085 1972 raft_consensus.cc:3037] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 8c61d200e5c3498baaf9c99081547b0a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:44.071059 2198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f89dc2a9a0e642b4b40ff3dba189c9c6" candidate_uuid: "32130acdc9dd4e82b1afcba7f40682d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02eb86b727b9472db54078654d5742bd"
I20260811 18:45:44.071259 2198 raft_consensus.cc:3037] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 02eb86b727b9472db54078654d5742bd [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:44.073319 1972 raft_consensus.cc:2454] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 8c61d200e5c3498baaf9c99081547b0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32130acdc9dd4e82b1afcba7f40682d1 in term 1.
I20260811 18:45:44.073580 2198 raft_consensus.cc:2454] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 02eb86b727b9472db54078654d5742bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32130acdc9dd4e82b1afcba7f40682d1 in term 1.
I20260811 18:45:44.073696 1587 leader_election.cc:302] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32130acdc9dd4e82b1afcba7f40682d1, 8c61d200e5c3498baaf9c99081547b0a; no voters:
I20260811 18:45:44.074026 2328 raft_consensus.cc:2781] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:45:44.074484 2328 raft_consensus.cc:686] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1 [term 1 LEADER]: Becoming Leader. State: Replica: 32130acdc9dd4e82b1afcba7f40682d1, State: Running, Role: LEADER
I20260811 18:45:44.074963 2328 consensus_queue.cc:237] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61d200e5c3498baaf9c99081547b0a" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 42433 } } peers { permanent_uuid: "32130acdc9dd4e82b1afcba7f40682d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 46683 } } peers { permanent_uuid: "02eb86b727b9472db54078654d5742bd" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42915 } }
I20260811 18:45:44.079644 1451 catalog_manager.cc:5495] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32130acdc9dd4e82b1afcba7f40682d1 (127.30.185.129). New cstate: current_term: 1 leader_uuid: "32130acdc9dd4e82b1afcba7f40682d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61d200e5c3498baaf9c99081547b0a" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 42433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32130acdc9dd4e82b1afcba7f40682d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 46683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02eb86b727b9472db54078654d5742bd" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42915 } health_report { overall_health: UNKNOWN } } }
I20260811 18:45:44.118772 2315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:44.113188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40348 (local address 127.30.185.129:46683)
0811 18:45:44.113408 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:44.113414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:44.113435 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:45:44.113652 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:44.113656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:44.113741 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:44.113895 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:45:44.114294 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:44.115011 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:44.116446 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:44.116870 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:44.117006 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:44.117713 (+ 707us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:45:44.118186 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:45:44.118389 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:44.118506 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 18:45:44.118550 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 18:45:44.127048 1972 raft_consensus.cc:1261] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 8c61d200e5c3498baaf9c99081547b0a [term 1 FOLLOWER]: Refusing update from remote peer 32130acdc9dd4e82b1afcba7f40682d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:45:44.127048 2198 raft_consensus.cc:1261] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 02eb86b727b9472db54078654d5742bd [term 1 FOLLOWER]: Refusing update from remote peer 32130acdc9dd4e82b1afcba7f40682d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:45:44.128134 2332 consensus_queue.cc:1035] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02eb86b727b9472db54078654d5742bd" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 42915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:45:44.128537 2328 consensus_queue.cc:1035] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61d200e5c3498baaf9c99081547b0a" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 42433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:45:44.140998 2340 mvcc.cc:205] Tried to move back new op lower bound from 7317397275133628416 to 7317397274934644736. Current Snapshot: MvccSnapshot[applied={T|T < 7317397275133628416}]
I20260811 18:45:44.154929 1451 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:45:44.155225 1451 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:45:44.159093 1451 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27373c59d27745b299f91fe567d36db6: Sending DeleteTablet for 3 replicas of tablet f89dc2a9a0e642b4b40ff3dba189c9c6
I20260811 18:45:44.160408 1671 tablet_service.cc:1514] Processing DeleteTablet for tablet f89dc2a9a0e642b4b40ff3dba189c9c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:45:44 UTC) from {username='slave'} at 127.0.0.1:40342
I20260811 18:45:44.160705 1938 tablet_service.cc:1514] Processing DeleteTablet for tablet f89dc2a9a0e642b4b40ff3dba189c9c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:45:44 UTC) from {username='slave'} at 127.0.0.1:40966
I20260811 18:45:44.160705 2166 tablet_service.cc:1514] Processing DeleteTablet for tablet f89dc2a9a0e642b4b40ff3dba189c9c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:45:44 UTC) from {username='slave'} at 127.0.0.1:56796
I20260811 18:45:44.162675 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 1557
I20260811 18:45:44.170424 2363 tablet_replica.cc:330] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 8c61d200e5c3498baaf9c99081547b0a: stopping tablet replica
I20260811 18:45:44.171723 2363 raft_consensus.cc:2229] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 8c61d200e5c3498baaf9c99081547b0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:45:44.173154 2363 raft_consensus.cc:2258] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 8c61d200e5c3498baaf9c99081547b0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:45:44.176822 2363 ts_tablet_manager.cc:1905] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 8c61d200e5c3498baaf9c99081547b0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:45:44.178824 2367 tablet_replica.cc:330] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 02eb86b727b9472db54078654d5742bd: stopping tablet replica
I20260811 18:45:44.179922 2367 raft_consensus.cc:2229] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 02eb86b727b9472db54078654d5742bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:45:44.181072 2367 raft_consensus.cc:2258] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 02eb86b727b9472db54078654d5742bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:45:44.184120 2363 ts_tablet_manager.cc:1918] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 8c61d200e5c3498baaf9c99081547b0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:45:44.184298 2363 log.cc:1196] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 8c61d200e5c3498baaf9c99081547b0a: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/f89dc2a9a0e642b4b40ff3dba189c9c6
I20260811 18:45:44.185017 2361 tablet_replica.cc:330] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1: stopping tablet replica
I20260811 18:45:44.185019 2367 ts_tablet_manager.cc:1905] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 02eb86b727b9472db54078654d5742bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:45:44.185070 2363 ts_tablet_manager.cc:1939] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 8c61d200e5c3498baaf9c99081547b0a: Deleting consensus metadata
I20260811 18:45:44.187054 1432 catalog_manager.cc:4841] TS 8c61d200e5c3498baaf9c99081547b0a (127.30.185.130:42433): tablet f89dc2a9a0e642b4b40ff3dba189c9c6 (table test-table [id=fd2d7ab3a622437a9240f79488ba68a0]) successfully deleted
I20260811 18:45:44.187443 2361 raft_consensus.cc:2229] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:45:44.188105 2361 raft_consensus.cc:2258] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:45:44.189016 2315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:44.166030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40360 (local address 127.30.185.129:46683)
0811 18:45:44.166289 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:44.166298 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:44.177560 (+ 11262us) server_negotiation.cc:407] Connection header received
0811 18:45:44.177896 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:44.177905 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:44.178037 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:44.178147 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:44.178959 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:44.180319 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:44.182547 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:44.183251 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:44.183365 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:44.185836 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:44.185848 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:44.185865 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:44.186643 (+ 778us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:44.186704 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:44.186730 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:44.186814 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:44.188492 (+ 1678us) server_negotiation.cc:299] Negotiation successful
0811 18:45:44.188635 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 18:45:44.190287 1791 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:44.192168 2367 ts_tablet_manager.cc:1918] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 02eb86b727b9472db54078654d5742bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:45:44.192397 2367 log.cc:1196] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 02eb86b727b9472db54078654d5742bd: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/f89dc2a9a0e642b4b40ff3dba189c9c6
I20260811 18:45:44.192958 2367 ts_tablet_manager.cc:1939] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 02eb86b727b9472db54078654d5742bd: Deleting consensus metadata
I20260811 18:45:44.194702 1428 catalog_manager.cc:4841] TS 02eb86b727b9472db54078654d5742bd (127.30.185.131:42915): tablet f89dc2a9a0e642b4b40ff3dba189c9c6 (table test-table [id=fd2d7ab3a622437a9240f79488ba68a0]) successfully deleted
I20260811 18:45:44.191493 2361 ts_tablet_manager.cc:1905] T f89dc2a9a0e642b4b40ff3dba189c9c6 P 32130acdc9dd4e82b1afcba7f40682d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:45:44.293898 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 1557
I20260811 18:45:44.328061 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 1806
W20260811 18:45:44.328056 1432 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.185.129:46683 (error 108)
I20260811 18:45:44.330271 2374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:44.329421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:46683 (local address 127.0.0.1:40374)
0811 18:45:44.329887 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:44.330067 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":241,"threads_started":1}
W20260811 18:45:44.330544 1432 catalog_manager.cc:4584] TS 32130acdc9dd4e82b1afcba7f40682d1 (127.30.185.129:46683): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f89dc2a9a0e642b4b40ff3dba189c9c6: Network error: Client connection negotiation failed: client connection to 127.30.185.129:46683: connect: Connection refused (error 111)
I20260811 18:45:44.336163 2303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:44.329037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40974 (local address 127.30.185.130:42433)
0811 18:45:44.329240 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:44.329246 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:44.329942 (+ 696us) server_negotiation.cc:407] Connection header received
0811 18:45:44.330147 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:44.330151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:44.330234 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:44.330319 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:44.331178 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:44.331923 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:44.333133 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:44.333665 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:44.333762 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:44.334751 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:44.334759 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:44.334783 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:44.335424 (+ 641us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:44.335471 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:44.335491 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:44.335576 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:44.335700 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 18:45:44.335798 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 18:45:44.336862 2040 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:44.384323 2374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:44.383835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:46683 (local address 127.0.0.1:40384)
0811 18:45:44.384073 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:44.384167 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260811 18:45:44.385665 2374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:44.385253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:46683 (local address 127.0.0.1:40390)
0811 18:45:44.385439 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:44.385515 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 18:45:44.428395 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 1806
I20260811 18:45:44.446156 2374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:44.445665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:46683 (local address 127.0.0.1:40398)
0811 18:45:44.445874 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:44.445983 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 18:45:44.447278 2374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:44.446902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:46683 (local address 127.0.0.1:40414)
0811 18:45:44.447077 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:44.447146 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 18:45:44.455345 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 2057
I20260811 18:45:44.462360 2306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:44.456147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56806 (local address 127.30.185.131:42915)
0811 18:45:44.456322 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:44.456327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:44.456347 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:45:44.456501 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:44.456506 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:44.456608 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:44.456693 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:44.457084 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:44.457894 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:44.459323 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:44.459722 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:44.459814 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:44.460926 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:44.460935 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:44.460951 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:44.461602 (+ 651us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:44.461659 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:44.461680 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:44.461771 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:44.461890 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 18:45:44.461980 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:45:44.463047 2274 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:44.528613 2374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:44.528151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:46683 (local address 127.0.0.1:40418)
0811 18:45:44.528371 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:44.528464 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 18:45:44.529762 2374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:44.529376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:46683 (local address 127.0.0.1:40422)
0811 18:45:44.529566 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:44.529635 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260811 18:45:44.553042 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 2057
I20260811 18:45:44.585565 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 1401
I20260811 18:45:44.586695 1497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:44.705806 2374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:44.705249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:46683 (local address 127.0.0.1:40436)
0811 18:45:44.705457 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:44.705592 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.129:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 18:45:44.705827 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 1401
2026-08-11T18:45:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3266 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:45:44 dist-test-slave-z8x9 krb5kdc[2382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:45:44 dist-test-slave-z8x9 krb5kdc[2382](info): set up 2 sockets
Aug 11 18:45:44 dist-test-slave-z8x9 krb5kdc[2382](info): commencing operation
krb5kdc: starting...
W20260811 18:45:46.129206 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.370s user 0.003s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:45:46 dist-test-slave-z8x9 krb5kdc[2382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:45:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:45:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:45:46.493875 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:43159
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:43159
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:45:46.846786 2406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:46.847147 2406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:46.847316 2406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:46.854604 2406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:45:46.854761 2406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:46.854882 2406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:46.854988 2406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:45:46.855067 2406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:45:46.865057 2406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:43159
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:43159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:46.866503 2406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:46.867862 2406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:46.879837 2413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:46.879848 2416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:46.879897 2414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:46.880622 2406 server_base.cc:1034] running on GCE node
I20260811 18:45:46.881534 2406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:46.882946 2406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:46.884197 2406 hybrid_clock.cc:648] HybridClock initialized: now 1786473946884155 us; error 53 us; skew 500 ppm
I20260811 18:45:46.884583 2406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:45:46 dist-test-slave-z8x9 krb5kdc[2382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473946, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:45:46.887885 2406 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:45:46.889441 2406 webserver.cc:460] Webserver started at http://127.30.185.190:39947/ using document root <none> and password file <none>
I20260811 18:45:46.890156 2406 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:46.890312 2406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:46.890625 2406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:46.893745 2406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "abc77adbd87d4e16bc7dcf0958f8652a"
format_stamp: "Formatted at 2026-08-11 18:45:46 on dist-test-slave-z8x9"
server_key: "d9717b9bb643e43ba5eef99e524d02c7"
server_key_iv: "00831291072c36dc1b399f44d3200175"
server_key_version: "encryptionkey@0"
I20260811 18:45:46.894531 2406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "abc77adbd87d4e16bc7dcf0958f8652a"
format_stamp: "Formatted at 2026-08-11 18:45:46 on dist-test-slave-z8x9"
server_key: "d9717b9bb643e43ba5eef99e524d02c7"
server_key_iv: "00831291072c36dc1b399f44d3200175"
server_key_version: "encryptionkey@0"
I20260811 18:45:46.899820 2406 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 18:45:46.903357 2429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:46.904752 2406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:45:46.904955 2406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "abc77adbd87d4e16bc7dcf0958f8652a"
format_stamp: "Formatted at 2026-08-11 18:45:46 on dist-test-slave-z8x9"
server_key: "d9717b9bb643e43ba5eef99e524d02c7"
server_key_iv: "00831291072c36dc1b399f44d3200175"
server_key_version: "encryptionkey@0"
I20260811 18:45:46.905174 2406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:46.920962 2406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:46.925740 2406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:46.926062 2406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:46.945772 2406 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:43159
I20260811 18:45:46.945797 2519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:43159 every 8 connection(s)
I20260811 18:45:46.947249 2406 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:45:46.949003 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 2406
I20260811 18:45:46.949229 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:45:46.949721 31462 external_mini_cluster.cc:1428] Setting key f35b51b19c69ce118fc4d3b4786728ed
I20260811 18:45:46.951532 2523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:46.965574 2523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a: Bootstrap starting.
Aug 11 18:45:46 dist-test-slave-z8x9 krb5kdc[2382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:45:46.969029 2523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:46.970417 2523 log.cc:824] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:46.973783 2523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a: No bootstrap required, opened a new log
I20260811 18:45:46.978078 2526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:46.951631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60888 (local address 127.30.185.190:43159)
0811 18:45:46.952461 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:46.952478 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:46.952535 (+ 57us) server_negotiation.cc:407] Connection header received
0811 18:45:46.954772 (+ 2237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:46.954799 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:46.955320 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:46.955764 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:46.957396 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:46.959495 (+ 2099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:46.961008 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:46.962149 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:46.962382 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:46.969759 (+ 7377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:46.969771 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:46.969790 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:46.969848 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:46.973493 (+ 3645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:46.974513 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:46.974521 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:46.974528 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:46.974709 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:46.975168 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:46.975171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:46.975173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:46.975716 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:46.975962 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:46.976461 (+ 499us) server_negotiation.cc:299] Negotiation successful
0811 18:45:46.976857 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":332,"threads_started":1}
I20260811 18:45:46.982487 2523 raft_consensus.cc:348] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abc77adbd87d4e16bc7dcf0958f8652a" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43159 } }
I20260811 18:45:46.982887 2523 raft_consensus.cc:374] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:46.983000 2523 raft_consensus.cc:729] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abc77adbd87d4e16bc7dcf0958f8652a, State: Initialized, Role: FOLLOWER
I20260811 18:45:46.983544 2523 consensus_queue.cc:260] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abc77adbd87d4e16bc7dcf0958f8652a" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43159 } }
I20260811 18:45:46.983791 2523 raft_consensus.cc:388] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:45:46.983934 2523 raft_consensus.cc:482] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:45:46.984112 2523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:46.986418 2523 raft_consensus.cc:504] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abc77adbd87d4e16bc7dcf0958f8652a" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43159 } }
I20260811 18:45:46.986925 2523 leader_election.cc:302] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abc77adbd87d4e16bc7dcf0958f8652a; no voters:
I20260811 18:45:46.987721 2523 leader_election.cc:288] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:45:46.987887 2528 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:45:46.988986 2528 raft_consensus.cc:686] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a [term 1 LEADER]: Becoming Leader. State: Replica: abc77adbd87d4e16bc7dcf0958f8652a, State: Running, Role: LEADER
I20260811 18:45:46.989537 2528 consensus_queue.cc:237] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abc77adbd87d4e16bc7dcf0958f8652a" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43159 } }
I20260811 18:45:46.989800 2523 sys_catalog.cc:564] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:45:46.992379 2529 sys_catalog.cc:455] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abc77adbd87d4e16bc7dcf0958f8652a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abc77adbd87d4e16bc7dcf0958f8652a" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43159 } } }
I20260811 18:45:46.992820 2529 sys_catalog.cc:458] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a [sys.catalog]: This master's current role is: LEADER
I20260811 18:45:46.992435 2535 sys_catalog.cc:455] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a [sys.catalog]: SysCatalogTable state changed. Reason: New leader abc77adbd87d4e16bc7dcf0958f8652a. Latest consensus state: current_term: 1 leader_uuid: "abc77adbd87d4e16bc7dcf0958f8652a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abc77adbd87d4e16bc7dcf0958f8652a" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43159 } } }
I20260811 18:45:46.994205 2535 sys_catalog.cc:458] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a [sys.catalog]: This master's current role is: LEADER
I20260811 18:45:46.995935 2545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:45:47.002106 2545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:45:47.011742 2545 catalog_manager.cc:1347] Generated new cluster ID: f828a70f93e24cd89218fcd7664375ea
I20260811 18:45:47.011895 2545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:45:47.022244 2545 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:45:47.023646 2545 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:45:47.030259 2545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a: Generated new TSK 0
I20260811 18:45:47.031325 2545 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.185.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
Entry for principal kudu/127.30.185.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab.
I20260811 18:45:47.112226 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:43159
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:45:47.462790 2572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:47.463099 2572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:47.463196 2572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:47.470877 2572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:47.471000 2572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:47.471114 2572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:45:47.481695 2572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.129.keytab
--principal=kudu/127.30.185.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:43159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:47.482960 2572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:47.484230 2572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:47.498478 2581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:47.498452 2582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:47.498558 2587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:47.499306 2572 server_base.cc:1034] running on GCE node
I20260811 18:45:47.499995 2572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:47.500964 2572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:47.502204 2572 hybrid_clock.cc:648] HybridClock initialized: now 1786473947502150 us; error 63 us; skew 500 ppm
I20260811 18:45:47.502550 2572 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:45:47 dist-test-slave-z8x9 krb5kdc[2382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473947, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:45:47.506026 2572 init.cc:377] Logged in from keytab as kudu/127.30.185.129@KRBTEST.COM (short username kudu)
I20260811 18:45:47.507617 2572 webserver.cc:460] Webserver started at http://127.30.185.129:44515/ using document root <none> and password file <none>
I20260811 18:45:47.508430 2572 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:47.508574 2572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:47.508929 2572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:47.512094 2572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "7551781f67384da28b359dfd31842f2b"
format_stamp: "Formatted at 2026-08-11 18:45:47 on dist-test-slave-z8x9"
server_key: "ab498a81f6114f9c3acd6f51c95506c3"
server_key_iv: "4b76542101cdb4300c82ab3b1164b0b5"
server_key_version: "encryptionkey@0"
I20260811 18:45:47.512899 2572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "7551781f67384da28b359dfd31842f2b"
format_stamp: "Formatted at 2026-08-11 18:45:47 on dist-test-slave-z8x9"
server_key: "ab498a81f6114f9c3acd6f51c95506c3"
server_key_iv: "4b76542101cdb4300c82ab3b1164b0b5"
server_key_version: "encryptionkey@0"
I20260811 18:45:47.518249 2572 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 18:45:47.521809 2603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:47.523141 2572 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 18:45:47.523331 2572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "7551781f67384da28b359dfd31842f2b"
format_stamp: "Formatted at 2026-08-11 18:45:47 on dist-test-slave-z8x9"
server_key: "ab498a81f6114f9c3acd6f51c95506c3"
server_key_iv: "4b76542101cdb4300c82ab3b1164b0b5"
server_key_version: "encryptionkey@0"
I20260811 18:45:47.523535 2572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:47.544986 2572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:47.549113 2572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:47.549458 2572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:47.550444 2572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:45:47.551753 2572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:45:47.551834 2572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:47.552004 2572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:45:47.552065 2572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:47.586751 2572 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:34489
I20260811 18:45:47.586807 2801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:34489 every 8 connection(s)
I20260811 18:45:47.588100 2572 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:45:47.596989 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 2572
I20260811 18:45:47.597234 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:45:47.597715 31462 external_mini_cluster.cc:1428] Setting key 8163a0abdc3b65b610e7457be37f2ce9
Aug 11 18:45:47 dist-test-slave-z8x9 krb5kdc[2382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473947, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.129@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:45:47.611151 2808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:47.590811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:55265 (local address 127.30.185.190:43159)
0811 18:45:47.591267 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:47.591272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:47.592243 (+ 971us) server_negotiation.cc:407] Connection header received
0811 18:45:47.594239 (+ 1996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:47.594245 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:47.594343 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:47.594467 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:47.596128 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:47.597105 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:47.599095 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:47.599640 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:47.599758 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:47.603664 (+ 3906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:47.603673 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:47.603681 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:47.603729 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:47.607479 (+ 3750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:47.608267 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:47.608276 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:47.608281 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:47.608448 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:47.608956 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:47.608965 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:47.608970 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:47.609415 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:47.609572 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:47.610408 (+ 836us) server_negotiation.cc:299] Negotiation successful
0811 18:45:47.610673 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":244,"threads_started":1}
I20260811 18:45:47.611894 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:47.591357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:43159 (local address 127.30.185.129:55265)
0811 18:45:47.592076 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:47.592121 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:47.593954 (+ 1833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:47.594710 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:47.594719 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:47.595266 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:45:47.595948 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:47.595967 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:47.597334 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:47.597342 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:47.598954 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:47.598969 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:47.599882 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:47.599888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:47.600091 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:47.600844 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:45:47.600876 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:45:47.603411 (+ 2535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:45:47.607639 (+ 4228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:45:47.607647 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:45:47.607661 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:45:47.608140 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:45:47.608561 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:45:47.608564 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:45:47.608567 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:45:47.608824 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:45:47.609580 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:45:47.609586 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:45:47.609999 (+ 413us) client_negotiation.cc:769] Sending connection context
0811 18:45:47.610247 (+ 248us) client_negotiation.cc:240] Negotiation successful
0811 18:45:47.610661 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":281,"threads_started":1}
I20260811 18:45:47.613560 2804 heartbeater.cc:347] Connected to a master server at 127.30.185.190:43159
I20260811 18:45:47.613906 2804 heartbeater.cc:464] Registering TS with master...
I20260811 18:45:47.614820 2804 heartbeater.cc:511] Master 127.30.185.190:43159 requested a full tablet report, sending...
I20260811 18:45:47.617286 2457 ts_manager.cc:194] Registered new tserver with Master: 7551781f67384da28b359dfd31842f2b (127.30.185.129:34489)
I20260811 18:45:47.619526 2457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.129@KRBTEST.COM'} at 127.30.185.129:55265
WARNING: no policy specified for kudu/127.30.185.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
Entry for principal kudu/127.30.185.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab.
I20260811 18:45:47.672930 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:43159
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:45:48.026028 2816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:48.026345 2816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:48.026533 2816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:48.034461 2816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:48.034612 2816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:48.034806 2816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:45:48.044950 2816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.130.keytab
--principal=kudu/127.30.185.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:43159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:48.046372 2816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:48.047732 2816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:48.061607 2827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:48.061690 2829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:48.061666 2824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:48.062675 2816 server_base.cc:1034] running on GCE node
I20260811 18:45:48.063344 2816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:48.064376 2816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:48.065596 2816 hybrid_clock.cc:648] HybridClock initialized: now 1786473948065528 us; error 58 us; skew 500 ppm
I20260811 18:45:48.065963 2816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:45:48 dist-test-slave-z8x9 krb5kdc[2382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473948, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:45:48.069656 2816 init.cc:377] Logged in from keytab as kudu/127.30.185.130@KRBTEST.COM (short username kudu)
I20260811 18:45:48.071431 2816 webserver.cc:460] Webserver started at http://127.30.185.130:44809/ using document root <none> and password file <none>
I20260811 18:45:48.072184 2816 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:48.072345 2816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:48.072683 2816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:48.075807 2816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "0694575513f4481bb788d4e08f125425"
format_stamp: "Formatted at 2026-08-11 18:45:48 on dist-test-slave-z8x9"
server_key: "bd732f93e36b67af2f0a17b9dcb43124"
server_key_iv: "a6502f0ee140bd519494325783c88203"
server_key_version: "encryptionkey@0"
I20260811 18:45:48.076628 2816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "0694575513f4481bb788d4e08f125425"
format_stamp: "Formatted at 2026-08-11 18:45:48 on dist-test-slave-z8x9"
server_key: "bd732f93e36b67af2f0a17b9dcb43124"
server_key_iv: "a6502f0ee140bd519494325783c88203"
server_key_version: "encryptionkey@0"
I20260811 18:45:48.082542 2816 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 18:45:48.086364 2847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:48.087755 2816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:45:48.087967 2816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "0694575513f4481bb788d4e08f125425"
format_stamp: "Formatted at 2026-08-11 18:45:48 on dist-test-slave-z8x9"
server_key: "bd732f93e36b67af2f0a17b9dcb43124"
server_key_iv: "a6502f0ee140bd519494325783c88203"
server_key_version: "encryptionkey@0"
I20260811 18:45:48.088235 2816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:48.109417 2816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:48.114215 2816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:48.114581 2816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:48.115645 2816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:45:48.117142 2816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:45:48.117316 2816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 18:45:48.117489 2816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:45:48.117568 2816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:48.153805 2816 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:39899
I20260811 18:45:48.153858 3041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:39899 every 8 connection(s)
I20260811 18:45:48.155265 2816 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:45:48.157368 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 2816
I20260811 18:45:48.157600 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:45:48.158270 31462 external_mini_cluster.cc:1428] Setting key 975905b9c9414d8505203d93f69e1b0e
Aug 11 18:45:48 dist-test-slave-z8x9 krb5kdc[2382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473948, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.130@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:45:48.179179 3046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:48.158034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:39667 (local address 127.30.185.190:43159)
0811 18:45:48.158543 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:48.158548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:48.159979 (+ 1431us) server_negotiation.cc:407] Connection header received
0811 18:45:48.162071 (+ 2092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:48.162079 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:48.162171 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:48.162305 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:48.163963 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:48.165118 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:48.166865 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:48.167658 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:48.167801 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:48.171939 (+ 4138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:48.171949 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:48.171957 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:48.172007 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:48.175484 (+ 3477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:48.176617 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:48.176621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:48.176624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:48.176752 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:48.177336 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:48.177340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:48.177342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:48.177626 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:48.177741 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:48.178731 (+ 990us) server_negotiation.cc:299] Negotiation successful
0811 18:45:48.178906 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":347,"threads_started":1}
I20260811 18:45:48.181020 3050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:48.158774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:43159 (local address 127.30.185.130:39667)
0811 18:45:48.159800 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:48.159851 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:48.161782 (+ 1931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:48.162531 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:48.162540 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:48.163151 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:45:48.163772 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:48.163791 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:48.165311 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:48.165315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:48.166683 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:48.166696 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:48.167904 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:48.167910 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:48.168143 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:48.168988 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:45:48.169019 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:45:48.171659 (+ 2640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:45:48.175654 (+ 3995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:45:48.175664 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:45:48.175689 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:45:48.176486 (+ 797us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:45:48.176871 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:45:48.176879 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:45:48.176886 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:45:48.177220 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:45:48.177762 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:45:48.177775 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:45:48.178312 (+ 537us) client_negotiation.cc:769] Sending connection context
0811 18:45:48.178655 (+ 343us) client_negotiation.cc:240] Negotiation successful
0811 18:45:48.179339 (+ 684us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":437,"threads_started":1}
I20260811 18:45:48.182648 3044 heartbeater.cc:347] Connected to a master server at 127.30.185.190:43159
I20260811 18:45:48.182973 3044 heartbeater.cc:464] Registering TS with master...
I20260811 18:45:48.183718 3044 heartbeater.cc:511] Master 127.30.185.190:43159 requested a full tablet report, sending...
I20260811 18:45:48.185125 2457 ts_manager.cc:194] Registered new tserver with Master: 0694575513f4481bb788d4e08f125425 (127.30.185.130:39899)
I20260811 18:45:48.186012 2457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.130@KRBTEST.COM'} at 127.30.185.130:39667
WARNING: no policy specified for kudu/127.30.185.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
Entry for principal kudu/127.30.185.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab.
I20260811 18:45:48.236275 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:43159
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:45:48.587559 3057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:48.587885 3057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:48.588047 3057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:48.595872 3057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:48.596024 3057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:48.596158 3057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:45:48.606263 3057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.131.keytab
--principal=kudu/127.30.185.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:43159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:48.607587 3057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:48.608872 3057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:48.622105 3064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:48.622062 3069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:48.622159 3065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:48.622711 2804 heartbeater.cc:503] Master 127.30.185.190:43159 was elected leader, sending a full tablet report...
I20260811 18:45:48.623029 3057 server_base.cc:1034] running on GCE node
I20260811 18:45:48.623683 3057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:48.624678 3057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:48.625891 3057 hybrid_clock.cc:648] HybridClock initialized: now 1786473948625835 us; error 49 us; skew 500 ppm
I20260811 18:45:48.626156 3057 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:45:48 dist-test-slave-z8x9 krb5kdc[2382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473948, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:45:48.629513 3057 init.cc:377] Logged in from keytab as kudu/127.30.185.131@KRBTEST.COM (short username kudu)
I20260811 18:45:48.630990 3057 webserver.cc:460] Webserver started at http://127.30.185.131:35425/ using document root <none> and password file <none>
I20260811 18:45:48.631680 3057 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:48.631829 3057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:48.632153 3057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:48.635195 3057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "d77737f36c0b423383c082744c7dd3aa"
format_stamp: "Formatted at 2026-08-11 18:45:48 on dist-test-slave-z8x9"
server_key: "bcb9902428fe2292d7898899b9a487cd"
server_key_iv: "0bd1f2fc093009e56ecde8c8cfc196de"
server_key_version: "encryptionkey@0"
I20260811 18:45:48.635989 3057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "d77737f36c0b423383c082744c7dd3aa"
format_stamp: "Formatted at 2026-08-11 18:45:48 on dist-test-slave-z8x9"
server_key: "bcb9902428fe2292d7898899b9a487cd"
server_key_iv: "0bd1f2fc093009e56ecde8c8cfc196de"
server_key_version: "encryptionkey@0"
I20260811 18:45:48.641433 3057 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 18:45:48.645310 3080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:48.646586 3057 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:45:48.646770 3057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "d77737f36c0b423383c082744c7dd3aa"
format_stamp: "Formatted at 2026-08-11 18:45:48 on dist-test-slave-z8x9"
server_key: "bcb9902428fe2292d7898899b9a487cd"
server_key_iv: "0bd1f2fc093009e56ecde8c8cfc196de"
server_key_version: "encryptionkey@0"
I20260811 18:45:48.646997 3057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:48.671290 3057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:48.675894 3057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:48.676195 3057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:48.677264 3057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:45:48.678650 3057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:45:48.678759 3057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:48.678906 3057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:45:48.678977 3057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:48.713667 3057 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:46507
I20260811 18:45:48.713712 3295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:46507 every 8 connection(s)
I20260811 18:45:48.715031 3057 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:45:48.721719 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 3057
I20260811 18:45:48.721932 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:45:48.722370 31462 external_mini_cluster.cc:1428] Setting key 9693ba0e02d408b8fda3a2b3938eade7
Aug 11 18:45:48 dist-test-slave-z8x9 krb5kdc[2382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473948, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.131@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:45:48.735368 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:48.717719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:50023 (local address 127.30.185.190:43159)
0811 18:45:48.718196 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:48.718200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:48.719196 (+ 996us) server_negotiation.cc:407] Connection header received
0811 18:45:48.721119 (+ 1923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:48.721125 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:48.721260 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:48.721380 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:48.723041 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:48.723904 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:48.725502 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:48.725925 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:48.726063 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:48.729610 (+ 3547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:48.729616 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:48.729619 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:48.729650 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:48.732259 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:48.732979 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:48.732987 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:48.732991 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:48.733116 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:48.733572 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:48.733577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:48.733580 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:48.733856 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:48.734031 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:48.734702 (+ 671us) server_negotiation.cc:299] Negotiation successful
0811 18:45:48.734955 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":235,"threads_started":1}
I20260811 18:45:48.736482 3301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:48.718300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:43159 (local address 127.30.185.131:50023)
0811 18:45:48.719022 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:48.719065 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:48.720849 (+ 1784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:48.721634 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:48.721642 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:48.722257 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:45:48.722862 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:48.722885 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:48.724068 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:48.724072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:48.725361 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:48.725374 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:48.726164 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:48.726174 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:48.726400 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:48.727105 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:45:48.727128 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:45:48.729402 (+ 2274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:45:48.732410 (+ 3008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:45:48.732419 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:45:48.732435 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:45:48.732862 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:45:48.733253 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:45:48.733259 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:45:48.733263 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:45:48.733459 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:45:48.733990 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:45:48.733999 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:45:48.734337 (+ 338us) client_negotiation.cc:769] Sending connection context
0811 18:45:48.734610 (+ 273us) client_negotiation.cc:240] Negotiation successful
0811 18:45:48.735183 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":274,"threads_started":1}
I20260811 18:45:48.737993 3296 heartbeater.cc:347] Connected to a master server at 127.30.185.190:43159
I20260811 18:45:48.738304 3296 heartbeater.cc:464] Registering TS with master...
I20260811 18:45:48.738976 3296 heartbeater.cc:511] Master 127.30.185.190:43159 requested a full tablet report, sending...
I20260811 18:45:48.740413 2454 ts_manager.cc:194] Registered new tserver with Master: d77737f36c0b423383c082744c7dd3aa (127.30.185.131:46507)
I20260811 18:45:48.741382 2454 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.185.131@KRBTEST.COM'} at 127.30.185.131:50023
I20260811 18:45:48.749910 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:45:48.766537 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:48.754082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60900 (local address 127.30.185.190:43159)
0811 18:45:48.754330 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:48.754337 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:48.754627 (+ 290us) server_negotiation.cc:407] Connection header received
0811 18:45:48.754758 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:48.754762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:48.754824 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:48.754902 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:48.756313 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:48.757025 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:48.758365 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:48.758750 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:48.758843 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:48.761165 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:48.761210 (+ 45us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:48.761219 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:48.761252 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:48.763993 (+ 2741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:48.764658 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:48.764663 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:48.764665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:48.764785 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:48.765131 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:48.765135 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:48.765137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:48.765523 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:48.765714 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:48.766117 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 18:45:48.766287 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 18:45:48.770900 2454 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:45:48.777936 2454 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:45:48.814424 3321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:48.790147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:39899 (local address 127.0.0.1:50162)
0811 18:45:48.791593 (+ 1446us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:48.791643 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:48.791859 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:48.806445 (+ 14586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:48.806452 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:48.806478 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:48.806888 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:48.806902 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:48.808964 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:48.808972 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:48.811382 (+ 2410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:48.811398 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:48.813045 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:48.813061 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:48.813674 (+ 613us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:48.813720 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 18:45:48.813803 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 18:45:48.813952 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1033,"mutex_wait_us":417,"thread_start_us":2044,"threads_started":1}
I20260811 18:45:48.814776 3324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:48.792904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:46507 (local address 127.0.0.1:54950)
0811 18:45:48.804661 (+ 11757us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:48.804690 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:48.804870 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:48.807676 (+ 2806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:48.807681 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:48.807708 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:48.808059 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:48.808070 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:48.810402 (+ 2332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:48.810408 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:48.812085 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:48.812101 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:48.813981 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:48.813988 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:48.814173 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:48.814204 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 18:45:48.814320 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 18:45:48.814451 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11551,"thread_start_us":12299,"threads_started":1}
I20260811 18:45:48.814885 3323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:48.791136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47508 (local address 127.30.185.129:34489)
0811 18:45:48.804305 (+ 13169us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:48.804320 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:48.804369 (+ 49us) server_negotiation.cc:407] Connection header received
0811 18:45:48.804528 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:48.804541 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:48.804908 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:48.806359 (+ 1451us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:48.806656 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:48.808085 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:48.811557 (+ 3472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:48.812867 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:48.814150 (+ 1283us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:48.814325 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:48.814393 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 18:45:48.814558 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12904,"thread_start_us":14033,"threads_started":1}
I20260811 18:45:48.814926 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:48.793030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54950 (local address 127.30.185.131:46507)
0811 18:45:48.807068 (+ 14038us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:48.807076 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:48.807109 (+ 33us) server_negotiation.cc:407] Connection header received
0811 18:45:48.807207 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:48.807213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:48.807458 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:48.807653 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:48.808564 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:48.810279 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:48.812262 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:48.813657 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:48.813887 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:48.814035 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:48.814375 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 18:45:48.814568 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13868,"thread_start_us":13707,"threads_started":1}
I20260811 18:45:48.815404 3317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:48.790487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:34489 (local address 127.0.0.1:47508)
0811 18:45:48.792546 (+ 2059us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:48.792565 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:48.792669 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:48.805532 (+ 12863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:48.805540 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:48.805574 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:48.806114 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:48.806132 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:48.808236 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:48.808242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:48.811382 (+ 3140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:48.811397 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:48.813146 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:48.813153 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:48.813388 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:48.813448 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 18:45:48.813557 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 18:45:48.813754 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1925,"mutex_wait_us":49,"thread_start_us":818,"threads_started":1}
I20260811 18:45:48.815881 3318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:48.790394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50162 (local address 127.30.185.130:39899)
0811 18:45:48.805608 (+ 15214us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:48.805617 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:48.805649 (+ 32us) server_negotiation.cc:407] Connection header received
0811 18:45:48.805747 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:48.805758 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:48.805991 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:48.807055 (+ 1064us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:48.807282 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:48.808803 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:48.811593 (+ 2790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:48.812789 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:48.813346 (+ 557us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:48.813538 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:48.815361 (+ 1823us) server_negotiation.cc:299] Negotiation successful
0811 18:45:48.815548 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15033,"thread_start_us":15747,"threads_started":1}
I20260811 18:45:48.819149 2933 tablet_service.cc:1467] Processing CreateTablet for tablet 1b19b52864844b0ebc324e7cebc50dcc (DEFAULT_TABLE table=test-table [id=d0902379e07945b8b6c37695758a5d34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:45:48.819233 3174 tablet_service.cc:1467] Processing CreateTablet for tablet 1b19b52864844b0ebc324e7cebc50dcc (DEFAULT_TABLE table=test-table [id=d0902379e07945b8b6c37695758a5d34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:45:48.819233 2673 tablet_service.cc:1467] Processing CreateTablet for tablet 1b19b52864844b0ebc324e7cebc50dcc (DEFAULT_TABLE table=test-table [id=d0902379e07945b8b6c37695758a5d34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:45:48.820422 2933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b19b52864844b0ebc324e7cebc50dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:48.820607 2673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b19b52864844b0ebc324e7cebc50dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:48.820621 3174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b19b52864844b0ebc324e7cebc50dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:48.832309 3326 tablet_bootstrap.cc:492] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425: Bootstrap starting.
I20260811 18:45:48.835197 3329 tablet_bootstrap.cc:492] T 1b19b52864844b0ebc324e7cebc50dcc P d77737f36c0b423383c082744c7dd3aa: Bootstrap starting.
I20260811 18:45:48.837548 3326 tablet_bootstrap.cc:654] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:48.837971 3330 tablet_bootstrap.cc:492] T 1b19b52864844b0ebc324e7cebc50dcc P 7551781f67384da28b359dfd31842f2b: Bootstrap starting.
I20260811 18:45:48.839007 3329 tablet_bootstrap.cc:654] T 1b19b52864844b0ebc324e7cebc50dcc P d77737f36c0b423383c082744c7dd3aa: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:48.839159 3326 log.cc:824] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:48.840274 3329 log.cc:824] T 1b19b52864844b0ebc324e7cebc50dcc P d77737f36c0b423383c082744c7dd3aa: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:48.841408 3330 tablet_bootstrap.cc:654] T 1b19b52864844b0ebc324e7cebc50dcc P 7551781f67384da28b359dfd31842f2b: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:48.842538 3326 tablet_bootstrap.cc:492] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425: No bootstrap required, opened a new log
I20260811 18:45:48.842708 3330 log.cc:824] T 1b19b52864844b0ebc324e7cebc50dcc P 7551781f67384da28b359dfd31842f2b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:48.842854 3326 ts_tablet_manager.cc:1397] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260811 18:45:48.844041 3329 tablet_bootstrap.cc:492] T 1b19b52864844b0ebc324e7cebc50dcc P d77737f36c0b423383c082744c7dd3aa: No bootstrap required, opened a new log
I20260811 18:45:48.844348 3329 ts_tablet_manager.cc:1397] T 1b19b52864844b0ebc324e7cebc50dcc P d77737f36c0b423383c082744c7dd3aa: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 18:45:48.847335 3330 tablet_bootstrap.cc:492] T 1b19b52864844b0ebc324e7cebc50dcc P 7551781f67384da28b359dfd31842f2b: No bootstrap required, opened a new log
I20260811 18:45:48.847597 3330 ts_tablet_manager.cc:1397] T 1b19b52864844b0ebc324e7cebc50dcc P 7551781f67384da28b359dfd31842f2b: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 18:45:48.855161 3330 raft_consensus.cc:348] T 1b19b52864844b0ebc324e7cebc50dcc P 7551781f67384da28b359dfd31842f2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0694575513f4481bb788d4e08f125425" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39899 } } peers { permanent_uuid: "7551781f67384da28b359dfd31842f2b" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34489 } } peers { permanent_uuid: "d77737f36c0b423383c082744c7dd3aa" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 46507 } }
I20260811 18:45:48.855252 3329 raft_consensus.cc:348] T 1b19b52864844b0ebc324e7cebc50dcc P d77737f36c0b423383c082744c7dd3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0694575513f4481bb788d4e08f125425" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39899 } } peers { permanent_uuid: "7551781f67384da28b359dfd31842f2b" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34489 } } peers { permanent_uuid: "d77737f36c0b423383c082744c7dd3aa" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 46507 } }
I20260811 18:45:48.855235 3326 raft_consensus.cc:348] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0694575513f4481bb788d4e08f125425" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39899 } } peers { permanent_uuid: "7551781f67384da28b359dfd31842f2b" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34489 } } peers { permanent_uuid: "d77737f36c0b423383c082744c7dd3aa" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 46507 } }
I20260811 18:45:48.855679 3330 raft_consensus.cc:374] T 1b19b52864844b0ebc324e7cebc50dcc P 7551781f67384da28b359dfd31842f2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:48.855682 3329 raft_consensus.cc:374] T 1b19b52864844b0ebc324e7cebc50dcc P d77737f36c0b423383c082744c7dd3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:48.855790 3330 raft_consensus.cc:729] T 1b19b52864844b0ebc324e7cebc50dcc P 7551781f67384da28b359dfd31842f2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7551781f67384da28b359dfd31842f2b, State: Initialized, Role: FOLLOWER
I20260811 18:45:48.855794 3329 raft_consensus.cc:729] T 1b19b52864844b0ebc324e7cebc50dcc P d77737f36c0b423383c082744c7dd3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d77737f36c0b423383c082744c7dd3aa, State: Initialized, Role: FOLLOWER
I20260811 18:45:48.856209 3326 raft_consensus.cc:374] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:48.856333 3326 raft_consensus.cc:729] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0694575513f4481bb788d4e08f125425, State: Initialized, Role: FOLLOWER
I20260811 18:45:48.856458 3330 consensus_queue.cc:260] T 1b19b52864844b0ebc324e7cebc50dcc P 7551781f67384da28b359dfd31842f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0694575513f4481bb788d4e08f125425" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39899 } } peers { permanent_uuid: "7551781f67384da28b359dfd31842f2b" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34489 } } peers { permanent_uuid: "d77737f36c0b423383c082744c7dd3aa" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 46507 } }
I20260811 18:45:48.856572 3329 consensus_queue.cc:260] T 1b19b52864844b0ebc324e7cebc50dcc P d77737f36c0b423383c082744c7dd3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0694575513f4481bb788d4e08f125425" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39899 } } peers { permanent_uuid: "7551781f67384da28b359dfd31842f2b" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34489 } } peers { permanent_uuid: "d77737f36c0b423383c082744c7dd3aa" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 46507 } }
I20260811 18:45:48.857239 3326 consensus_queue.cc:260] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0694575513f4481bb788d4e08f125425" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39899 } } peers { permanent_uuid: "7551781f67384da28b359dfd31842f2b" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34489 } } peers { permanent_uuid: "d77737f36c0b423383c082744c7dd3aa" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 46507 } }
I20260811 18:45:48.858436 3330 ts_tablet_manager.cc:1428] T 1b19b52864844b0ebc324e7cebc50dcc P 7551781f67384da28b359dfd31842f2b: Time spent starting tablet: real 0.011s user 0.006s sys 0.006s
I20260811 18:45:48.859674 3044 heartbeater.cc:503] Master 127.30.185.190:43159 was elected leader, sending a full tablet report...
I20260811 18:45:48.860587 3326 ts_tablet_manager.cc:1428] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425: Time spent starting tablet: real 0.018s user 0.015s sys 0.001s
I20260811 18:45:48.861727 3296 heartbeater.cc:503] Master 127.30.185.190:43159 was elected leader, sending a full tablet report...
I20260811 18:45:48.862407 3329 ts_tablet_manager.cc:1428] T 1b19b52864844b0ebc324e7cebc50dcc P d77737f36c0b423383c082744c7dd3aa: Time spent starting tablet: real 0.018s user 0.014s sys 0.004s
W20260811 18:45:48.907397 3045 tablet.cc:2369] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:45:48.966826 3299 tablet.cc:2369] T 1b19b52864844b0ebc324e7cebc50dcc P d77737f36c0b423383c082744c7dd3aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:45:49.090898 2806 tablet.cc:2369] T 1b19b52864844b0ebc324e7cebc50dcc P 7551781f67384da28b359dfd31842f2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:45:49.119174 3342 raft_consensus.cc:482] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:45:49.119501 3342 raft_consensus.cc:504] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0694575513f4481bb788d4e08f125425" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39899 } } peers { permanent_uuid: "7551781f67384da28b359dfd31842f2b" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34489 } } peers { permanent_uuid: "d77737f36c0b423383c082744c7dd3aa" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 46507 } }
I20260811 18:45:49.121454 3342 leader_election.cc:288] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7551781f67384da28b359dfd31842f2b (127.30.185.129:34489), d77737f36c0b423383c082744c7dd3aa (127.30.185.131:46507)
I20260811 18:45:49.129319 3323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:49.121719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:52941 (local address 127.30.185.129:34489)
0811 18:45:49.122020 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:49.122028 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:49.122879 (+ 851us) server_negotiation.cc:407] Connection header received
0811 18:45:49.123108 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:49.123113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:49.123182 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:49.123293 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:49.123968 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:49.124761 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:49.127241 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:49.128212 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:49.128435 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:49.128525 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:49.128866 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 18:45:49.129007 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260811 18:45:49.129335 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:49.122068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:40105 (local address 127.30.185.131:46507)
0811 18:45:49.122361 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:49.122367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:49.122980 (+ 613us) server_negotiation.cc:407] Connection header received
0811 18:45:49.123180 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:49.123184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:49.123244 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:49.123344 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:49.124023 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:49.124759 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:49.127176 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:49.128295 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:49.128472 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:49.128563 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:49.128898 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 18:45:49.129031 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260811 18:45:49.129474 3345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:49.121979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:46507 (local address 127.30.185.130:40105)
0811 18:45:49.122856 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:49.122876 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:49.123027 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:49.123374 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:49.123379 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:49.123404 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:49.123845 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:49.123855 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:49.124914 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:49.124919 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:49.127021 (+ 2102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:49.127035 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:49.128573 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:49.128581 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:49.128733 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:49.128755 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 18:45:49.128866 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 18:45:49.128988 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"mutex_wait_us":47,"thread_start_us":555,"threads_started":1}
I20260811 18:45:49.129750 3344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:49.121583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:34489 (local address 127.30.185.130:52941)
0811 18:45:49.122726 (+ 1143us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:49.122756 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:49.122928 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:49.123309 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:49.123314 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:49.123351 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:49.123798 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:49.123808 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:49.124905 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:49.124914 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:49.127073 (+ 2159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:49.127086 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:49.128487 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:49.128494 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:49.128665 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:49.128695 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 18:45:49.128815 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 18:45:49.128960 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":970,"mutex_wait_us":40,"thread_start_us":711,"threads_started":1}
I20260811 18:45:49.130606 3211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b19b52864844b0ebc324e7cebc50dcc" candidate_uuid: "0694575513f4481bb788d4e08f125425" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d77737f36c0b423383c082744c7dd3aa" is_pre_election: true
I20260811 18:45:49.130746 2712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b19b52864844b0ebc324e7cebc50dcc" candidate_uuid: "0694575513f4481bb788d4e08f125425" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7551781f67384da28b359dfd31842f2b" is_pre_election: true
I20260811 18:45:49.131137 3211 raft_consensus.cc:2454] T 1b19b52864844b0ebc324e7cebc50dcc P d77737f36c0b423383c082744c7dd3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0694575513f4481bb788d4e08f125425 in term 0.
I20260811 18:45:49.131304 2712 raft_consensus.cc:2454] T 1b19b52864844b0ebc324e7cebc50dcc P 7551781f67384da28b359dfd31842f2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0694575513f4481bb788d4e08f125425 in term 0.
I20260811 18:45:49.131879 2849 leader_election.cc:302] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0694575513f4481bb788d4e08f125425, d77737f36c0b423383c082744c7dd3aa; no voters:
I20260811 18:45:49.132303 3342 raft_consensus.cc:2781] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:45:49.132477 3342 raft_consensus.cc:482] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:45:49.132598 3342 raft_consensus.cc:3037] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:49.135041 3342 raft_consensus.cc:504] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0694575513f4481bb788d4e08f125425" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39899 } } peers { permanent_uuid: "7551781f67384da28b359dfd31842f2b" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34489 } } peers { permanent_uuid: "d77737f36c0b423383c082744c7dd3aa" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 46507 } }
I20260811 18:45:49.135746 3342 leader_election.cc:288] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425 [CANDIDATE]: Term 1 election: Requested vote from peers 7551781f67384da28b359dfd31842f2b (127.30.185.129:34489), d77737f36c0b423383c082744c7dd3aa (127.30.185.131:46507)
I20260811 18:45:49.136332 2712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b19b52864844b0ebc324e7cebc50dcc" candidate_uuid: "0694575513f4481bb788d4e08f125425" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7551781f67384da28b359dfd31842f2b"
I20260811 18:45:49.136345 3211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b19b52864844b0ebc324e7cebc50dcc" candidate_uuid: "0694575513f4481bb788d4e08f125425" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d77737f36c0b423383c082744c7dd3aa"
I20260811 18:45:49.136566 2712 raft_consensus.cc:3037] T 1b19b52864844b0ebc324e7cebc50dcc P 7551781f67384da28b359dfd31842f2b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:49.136565 3211 raft_consensus.cc:3037] T 1b19b52864844b0ebc324e7cebc50dcc P d77737f36c0b423383c082744c7dd3aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:49.139937 3211 raft_consensus.cc:2454] T 1b19b52864844b0ebc324e7cebc50dcc P d77737f36c0b423383c082744c7dd3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0694575513f4481bb788d4e08f125425 in term 1.
I20260811 18:45:49.139930 2712 raft_consensus.cc:2454] T 1b19b52864844b0ebc324e7cebc50dcc P 7551781f67384da28b359dfd31842f2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0694575513f4481bb788d4e08f125425 in term 1.
I20260811 18:45:49.140463 2853 leader_election.cc:302] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0694575513f4481bb788d4e08f125425, 7551781f67384da28b359dfd31842f2b; no voters:
I20260811 18:45:49.140796 3342 raft_consensus.cc:2781] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:45:49.141276 3342 raft_consensus.cc:686] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425 [term 1 LEADER]: Becoming Leader. State: Replica: 0694575513f4481bb788d4e08f125425, State: Running, Role: LEADER
I20260811 18:45:49.141799 3342 consensus_queue.cc:237] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0694575513f4481bb788d4e08f125425" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39899 } } peers { permanent_uuid: "7551781f67384da28b359dfd31842f2b" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34489 } } peers { permanent_uuid: "d77737f36c0b423383c082744c7dd3aa" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 46507 } }
I20260811 18:45:49.146914 2454 catalog_manager.cc:5495] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0694575513f4481bb788d4e08f125425 (127.30.185.130). New cstate: current_term: 1 leader_uuid: "0694575513f4481bb788d4e08f125425" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0694575513f4481bb788d4e08f125425" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 39899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7551781f67384da28b359dfd31842f2b" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d77737f36c0b423383c082744c7dd3aa" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 46507 } health_report { overall_health: UNKNOWN } } }
I20260811 18:45:49.182669 3318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:49.177579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50176 (local address 127.30.185.130:39899)
0811 18:45:49.177754 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:49.177758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:49.177776 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:45:49.177975 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:49.177978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:49.178057 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:49.178184 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:45:49.178528 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:49.179232 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:49.180585 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:49.180986 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:49.181079 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:49.181649 (+ 570us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:45:49.182153 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:45:49.182262 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:49.182391 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 18:45:49.182457 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 18:45:49.190797 2712 raft_consensus.cc:1261] T 1b19b52864844b0ebc324e7cebc50dcc P 7551781f67384da28b359dfd31842f2b [term 1 FOLLOWER]: Refusing update from remote peer 0694575513f4481bb788d4e08f125425: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:45:49.190797 3211 raft_consensus.cc:1261] T 1b19b52864844b0ebc324e7cebc50dcc P d77737f36c0b423383c082744c7dd3aa [term 1 FOLLOWER]: Refusing update from remote peer 0694575513f4481bb788d4e08f125425: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:45:49.191905 3342 consensus_queue.cc:1035] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d77737f36c0b423383c082744c7dd3aa" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:45:49.192219 3346 consensus_queue.cc:1035] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7551781f67384da28b359dfd31842f2b" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:45:49.202219 3353 mvcc.cc:205] Tried to move back new op lower bound from 7317397295875653632 to 7317397295688691712. Current Snapshot: MvccSnapshot[applied={T|T < 7317397295875653632}]
I20260811 18:45:49.205588 3355 mvcc.cc:205] Tried to move back new op lower bound from 7317397295875653632 to 7317397295688691712. Current Snapshot: MvccSnapshot[applied={T|T < 7317397295875653632}]
I20260811 18:45:49.224202 2454 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:45:49.224491 2454 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:45:49.228245 2454 catalog_manager.cc:5782] T 00000000000000000000000000000000 P abc77adbd87d4e16bc7dcf0958f8652a: Sending DeleteTablet for 3 replicas of tablet 1b19b52864844b0ebc324e7cebc50dcc
I20260811 18:45:49.229545 2673 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b19b52864844b0ebc324e7cebc50dcc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:45:49 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:47508
I20260811 18:45:49.229597 3174 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b19b52864844b0ebc324e7cebc50dcc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:45:49 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:54950
I20260811 18:45:49.230324 2933 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b19b52864844b0ebc324e7cebc50dcc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:45:49 UTC) from {username='kudu', principal='kudu/127.30.185.190@KRBTEST.COM'} at 127.0.0.1:50162
I20260811 18:45:49.230590 3368 tablet_replica.cc:330] T 1b19b52864844b0ebc324e7cebc50dcc P 7551781f67384da28b359dfd31842f2b: stopping tablet replica
I20260811 18:45:49.230686 3369 tablet_replica.cc:330] T 1b19b52864844b0ebc324e7cebc50dcc P d77737f36c0b423383c082744c7dd3aa: stopping tablet replica
I20260811 18:45:49.231091 3369 raft_consensus.cc:2229] T 1b19b52864844b0ebc324e7cebc50dcc P d77737f36c0b423383c082744c7dd3aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:45:49.231091 3368 raft_consensus.cc:2229] T 1b19b52864844b0ebc324e7cebc50dcc P 7551781f67384da28b359dfd31842f2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:45:49.231606 3369 raft_consensus.cc:2258] T 1b19b52864844b0ebc324e7cebc50dcc P d77737f36c0b423383c082744c7dd3aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:45:49.231604 3368 raft_consensus.cc:2258] T 1b19b52864844b0ebc324e7cebc50dcc P 7551781f67384da28b359dfd31842f2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:45:49.233884 3369 ts_tablet_manager.cc:1905] T 1b19b52864844b0ebc324e7cebc50dcc P d77737f36c0b423383c082744c7dd3aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:45:49.233884 3368 ts_tablet_manager.cc:1905] T 1b19b52864844b0ebc324e7cebc50dcc P 7551781f67384da28b359dfd31842f2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:45:49.235844 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 2572
I20260811 18:45:49.237644 3370 tablet_replica.cc:330] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425: stopping tablet replica
I20260811 18:45:49.238204 3370 raft_consensus.cc:2229] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:45:49.241063 3369 ts_tablet_manager.cc:1918] T 1b19b52864844b0ebc324e7cebc50dcc P d77737f36c0b423383c082744c7dd3aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:45:49.241256 3369 log.cc:1196] T 1b19b52864844b0ebc324e7cebc50dcc P d77737f36c0b423383c082744c7dd3aa: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/1b19b52864844b0ebc324e7cebc50dcc
I20260811 18:45:49.241818 3369 ts_tablet_manager.cc:1939] T 1b19b52864844b0ebc324e7cebc50dcc P d77737f36c0b423383c082744c7dd3aa: Deleting consensus metadata
I20260811 18:45:49.242619 3370 raft_consensus.cc:2258] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:45:49.244062 2431 catalog_manager.cc:4841] TS d77737f36c0b423383c082744c7dd3aa (127.30.185.131:46507): tablet 1b19b52864844b0ebc324e7cebc50dcc (table test-table [id=d0902379e07945b8b6c37695758a5d34]) successfully deleted
I20260811 18:45:49.246368 3368 ts_tablet_manager.cc:1918] T 1b19b52864844b0ebc324e7cebc50dcc P 7551781f67384da28b359dfd31842f2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:45:49.246570 3368 log.cc:1196] T 1b19b52864844b0ebc324e7cebc50dcc P 7551781f67384da28b359dfd31842f2b: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-0/wal/wals/1b19b52864844b0ebc324e7cebc50dcc
I20260811 18:45:49.247156 3368 ts_tablet_manager.cc:1939] T 1b19b52864844b0ebc324e7cebc50dcc P 7551781f67384da28b359dfd31842f2b: Deleting consensus metadata
I20260811 18:45:49.247715 3370 ts_tablet_manager.cc:1905] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:45:49.248997 2433 catalog_manager.cc:4841] TS 7551781f67384da28b359dfd31842f2b (127.30.185.129:34489): tablet 1b19b52864844b0ebc324e7cebc50dcc (table test-table [id=d0902379e07945b8b6c37695758a5d34]) successfully deleted
I20260811 18:45:49.259019 3370 ts_tablet_manager.cc:1918] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:45:49.259231 3370 log.cc:1196] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/1b19b52864844b0ebc324e7cebc50dcc
I20260811 18:45:49.259845 3370 ts_tablet_manager.cc:1939] T 1b19b52864844b0ebc324e7cebc50dcc P 0694575513f4481bb788d4e08f125425: Deleting consensus metadata
I20260811 18:45:49.261744 2431 catalog_manager.cc:4841] TS 0694575513f4481bb788d4e08f125425 (127.30.185.130:39899): tablet 1b19b52864844b0ebc324e7cebc50dcc (table test-table [id=d0902379e07945b8b6c37695758a5d34]) successfully deleted
Aug 11 18:45:49 dist-test-slave-z8x9 krb5kdc[2382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.129@KRBTEST.COM
I20260811 18:45:49.271332 3323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:49.237545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47522 (local address 127.30.185.129:34489)
0811 18:45:49.237731 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:49.237737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:49.243499 (+ 5762us) server_negotiation.cc:407] Connection header received
0811 18:45:49.244797 (+ 1298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:49.244803 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:49.244894 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:49.244991 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:49.249286 (+ 4295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:49.250361 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:49.255200 (+ 4839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:49.255870 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:49.256013 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:49.265448 (+ 9435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:49.265464 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:49.265479 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:49.265517 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:49.268412 (+ 2895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:49.269112 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:49.269119 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:49.269121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:49.269364 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:49.269884 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:49.269888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:49.269889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:49.270263 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:49.270497 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:49.270863 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 18:45:49.271034 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 18:45:49.272200 2794 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:49.364995 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 2572
I20260811 18:45:49.394903 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 2816
Aug 11 18:45:49 dist-test-slave-z8x9 krb5kdc[2382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.130@KRBTEST.COM
I20260811 18:45:49.411662 3318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:49.395755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50190 (local address 127.30.185.130:39899)
0811 18:45:49.395994 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:49.396001 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:49.396022 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:45:49.396167 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:49.396173 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:49.396252 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:49.396363 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:49.397991 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:49.398828 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:49.400140 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:49.400596 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:49.400731 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:49.405324 (+ 4593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:49.405339 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:49.405356 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:49.405390 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:49.408481 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:49.409408 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:49.409422 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:49.409425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:49.409582 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:49.410017 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:49.410024 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:49.410029 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:49.410406 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:49.410807 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:49.411022 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 18:45:49.411274 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260811 18:45:49.412388 3036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:49.528546 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 2816
I20260811 18:45:49.558951 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 3057
Aug 11 18:45:49 dist-test-slave-z8x9 krb5kdc[2382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.131@KRBTEST.COM
I20260811 18:45:49.573783 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:49.559722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54952 (local address 127.30.185.131:46507)
0811 18:45:49.559885 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:49.559890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:49.559958 (+ 68us) server_negotiation.cc:407] Connection header received
0811 18:45:49.560104 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:49.560108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:49.560182 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:49.560267 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:49.561675 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:49.562395 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:49.563619 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:49.564004 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:49.564094 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:49.568444 (+ 4350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:49.568460 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:49.568476 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:49.568509 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:49.571432 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:49.572026 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:49.572033 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:49.572036 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:49.572174 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:49.572496 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:49.572499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:49.572501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:49.572842 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:49.573050 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:49.573363 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 18:45:49.573525 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:45:49.574400 3286 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:49.663975 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 3057
I20260811 18:45:49.689905 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 2406
I20260811 18:45:49.690989 2514 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:49.803755 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 2406
2026-08-11T18:45:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5097 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T18:45:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:45:49Z Disabled control of system clock
I20260811 18:45:49.910101 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:43235
--webserver_interface=127.30.185.190
--webserver_port=0
--builtin_ntp_servers=127.30.185.148:35625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:43235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:45:50.266422 3388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:50.266736 3388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:50.266820 3388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:50.274194 3388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:45:50.274314 3388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:50.274354 3388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:50.274420 3388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:45:50.274464 3388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:45:50.283854 3388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:35625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:43235
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:43235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:50.285153 3388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:50.286966 3388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:50.299291 3404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:50.299264 3398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:50.299345 3394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:50.305544 3388 server_base.cc:1034] running on GCE node
I20260811 18:45:50.306460 3388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:50.308032 3388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:50.309273 3388 hybrid_clock.cc:648] HybridClock initialized: now 1786473950309210 us; error 52 us; skew 500 ppm
I20260811 18:45:50.309778 3388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:45:50.312966 3388 webserver.cc:460] Webserver started at http://127.30.185.190:36151/ using document root <none> and password file <none>
I20260811 18:45:50.313817 3388 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:50.313957 3388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:50.314306 3388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:50.317451 3388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "42dbcacc80734b0c8b502cc0dccd05a5"
format_stamp: "Formatted at 2026-08-11 18:45:50 on dist-test-slave-z8x9"
server_key: "9ad3a142b258f0f3a798a9d34aa86bf1"
server_key_iv: "dd2e4e4e8c28fe682e3690c79d3cafe6"
server_key_version: "encryptionkey@0"
I20260811 18:45:50.318312 3388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "42dbcacc80734b0c8b502cc0dccd05a5"
format_stamp: "Formatted at 2026-08-11 18:45:50 on dist-test-slave-z8x9"
server_key: "9ad3a142b258f0f3a798a9d34aa86bf1"
server_key_iv: "dd2e4e4e8c28fe682e3690c79d3cafe6"
server_key_version: "encryptionkey@0"
I20260811 18:45:50.323573 3388 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:45:50.327109 3413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:50.328462 3388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:45:50.328649 3388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "42dbcacc80734b0c8b502cc0dccd05a5"
format_stamp: "Formatted at 2026-08-11 18:45:50 on dist-test-slave-z8x9"
server_key: "9ad3a142b258f0f3a798a9d34aa86bf1"
server_key_iv: "dd2e4e4e8c28fe682e3690c79d3cafe6"
server_key_version: "encryptionkey@0"
I20260811 18:45:50.328909 3388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:50.342648 3388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:50.343497 3388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:50.343803 3388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:50.362354 3388 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:43235
I20260811 18:45:50.362376 3509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:43235 every 8 connection(s)
I20260811 18:45:50.363749 3388 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:45:50.367733 3511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:50.374007 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 3388
I20260811 18:45:50.374223 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:45:50.374817 31462 external_mini_cluster.cc:1428] Setting key b0f98b689872dad98db283f9608241db
I20260811 18:45:50.379683 3511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5: Bootstrap starting.
I20260811 18:45:50.382854 3511 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:50.383886 3511 log.cc:824] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:50.387018 3511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5: No bootstrap required, opened a new log
I20260811 18:45:50.388492 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:50.376967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49646 (local address 127.30.185.190:43235)
0811 18:45:50.378073 (+ 1106us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:50.378083 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:50.378125 (+ 42us) server_negotiation.cc:407] Connection header received
0811 18:45:50.379870 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:50.379895 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:50.380346 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:50.380709 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:50.381310 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:50.382573 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:50.383974 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:50.384746 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:50.384957 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:50.386209 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:50.386218 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:50.386231 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:50.386761 (+ 530us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:50.386848 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:50.386870 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:50.387001 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:50.387226 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 18:45:50.387552 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":425,"threads_started":1}
I20260811 18:45:50.396018 3511 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42dbcacc80734b0c8b502cc0dccd05a5" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43235 } }
I20260811 18:45:50.396426 3511 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:50.396535 3511 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42dbcacc80734b0c8b502cc0dccd05a5, State: Initialized, Role: FOLLOWER
I20260811 18:45:50.397112 3511 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42dbcacc80734b0c8b502cc0dccd05a5" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43235 } }
I20260811 18:45:50.397442 3511 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:45:50.397589 3511 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:45:50.397785 3511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:50.400033 3511 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42dbcacc80734b0c8b502cc0dccd05a5" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43235 } }
I20260811 18:45:50.400625 3511 leader_election.cc:302] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42dbcacc80734b0c8b502cc0dccd05a5; no voters:
I20260811 18:45:50.401316 3511 leader_election.cc:288] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:45:50.401480 3523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:45:50.402853 3523 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5 [term 1 LEADER]: Becoming Leader. State: Replica: 42dbcacc80734b0c8b502cc0dccd05a5, State: Running, Role: LEADER
I20260811 18:45:50.403393 3523 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42dbcacc80734b0c8b502cc0dccd05a5" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43235 } }
I20260811 18:45:50.403558 3511 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:45:50.406692 3526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42dbcacc80734b0c8b502cc0dccd05a5. Latest consensus state: current_term: 1 leader_uuid: "42dbcacc80734b0c8b502cc0dccd05a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42dbcacc80734b0c8b502cc0dccd05a5" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43235 } } }
I20260811 18:45:50.406838 3524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42dbcacc80734b0c8b502cc0dccd05a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42dbcacc80734b0c8b502cc0dccd05a5" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 43235 } } }
I20260811 18:45:50.407226 3526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5 [sys.catalog]: This master's current role is: LEADER
I20260811 18:45:50.407341 3524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5 [sys.catalog]: This master's current role is: LEADER
I20260811 18:45:50.409051 3544 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:45:50.413992 3544 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:45:50.422523 3544 catalog_manager.cc:1347] Generated new cluster ID: 1953d17194b5472d898c4c70ca3a51e5
I20260811 18:45:50.422652 3544 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:45:50.439651 3544 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:45:50.441562 3544 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:45:50.463001 3544 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5: Generated new TSK 0
I20260811 18:45:50.464238 3544 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:45:50.473764 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.129:0
--local_ip_for_outbound_sockets=127.30.185.129
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:43235
--builtin_ntp_servers=127.30.185.148:35625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:45:50.835839 3558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:50.836160 3558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:50.836254 3558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:50.843706 3558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:50.843833 3558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:50.843947 3558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.129
I20260811 18:45:50.853358 3558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:35625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.185.129
--webserver_port=0
--tserver_master_addrs=127.30.185.190:43235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.129
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:50.854686 3558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:50.856513 3558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:50.870771 3567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:50.870818 3571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:50.870872 3568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:50.878923 3558 server_base.cc:1034] running on GCE node
I20260811 18:45:50.879673 3558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:50.880765 3558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:50.881976 3558 hybrid_clock.cc:648] HybridClock initialized: now 1786473950881851 us; error 133 us; skew 500 ppm
I20260811 18:45:50.882330 3558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:45:50.884851 3558 webserver.cc:460] Webserver started at http://127.30.185.129:39695/ using document root <none> and password file <none>
I20260811 18:45:50.885695 3558 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:50.885977 3558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:50.886382 3558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:50.889578 3558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-0/data/instance:
uuid: "ffa9ad378d684546b13506d6633927d1"
format_stamp: "Formatted at 2026-08-11 18:45:50 on dist-test-slave-z8x9"
server_key: "8f84d4dd7ecee42880e0f9cffe150b28"
server_key_iv: "238d24d33906b1fb13ebf7974ae88264"
server_key_version: "encryptionkey@0"
I20260811 18:45:50.890429 3558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance:
uuid: "ffa9ad378d684546b13506d6633927d1"
format_stamp: "Formatted at 2026-08-11 18:45:50 on dist-test-slave-z8x9"
server_key: "8f84d4dd7ecee42880e0f9cffe150b28"
server_key_iv: "238d24d33906b1fb13ebf7974ae88264"
server_key_version: "encryptionkey@0"
I20260811 18:45:50.895895 3558 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:45:50.899571 3581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:50.900995 3558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:45:50.901240 3558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-0/wal
uuid: "ffa9ad378d684546b13506d6633927d1"
format_stamp: "Formatted at 2026-08-11 18:45:50 on dist-test-slave-z8x9"
server_key: "8f84d4dd7ecee42880e0f9cffe150b28"
server_key_iv: "238d24d33906b1fb13ebf7974ae88264"
server_key_version: "encryptionkey@0"
I20260811 18:45:50.901485 3558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:50.916930 3558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:50.917894 3558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:50.918210 3558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:50.919198 3558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:45:50.920717 3558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:45:50.920841 3558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:50.920965 3558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:45:50.921052 3558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:50.955231 3558 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.129:34779
I20260811 18:45:50.955289 3774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.129:34779 every 8 connection(s)
I20260811 18:45:50.956609 3558 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-0/data/info.pb
I20260811 18:45:50.959348 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 3558
I20260811 18:45:50.959553 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-0/wal/instance
I20260811 18:45:50.960100 31462 external_mini_cluster.cc:1428] Setting key a5aefef754e4ce02aacad3e5d43f2102
I20260811 18:45:50.963402 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.130:0
--local_ip_for_outbound_sockets=127.30.185.130
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:43235
--builtin_ntp_servers=127.30.185.148:35625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 18:45:50.971429 3777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:50.959359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:51471 (local address 127.30.185.190:43235)
0811 18:45:50.959860 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:50.959865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:50.961367 (+ 1502us) server_negotiation.cc:407] Connection header received
0811 18:45:50.963725 (+ 2358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:50.963734 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:50.963847 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:50.963966 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:50.965111 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:50.966358 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:50.968034 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:50.968747 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:50.968905 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:50.970424 (+ 1519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:50.970433 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:50.970440 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:50.970469 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:50.970524 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:50.970544 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:50.970639 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:50.970866 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 18:45:50.970983 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":289,"threads_started":1}
I20260811 18:45:50.972321 3780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:50.960038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:43235 (local address 127.30.185.129:51471)
0811 18:45:50.961125 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:50.961230 (+ 105us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:50.963353 (+ 2123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:50.964191 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:50.964199 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:50.964322 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:45:50.964924 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:50.964942 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:50.966524 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:50.966528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:50.967878 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:50.967891 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:50.968972 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:50.968977 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:50.969250 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:50.970049 (+ 799us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:45:50.970052 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:45:50.970096 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:45:50.970182 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:45:50.970652 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:45:50.970662 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:45:50.970667 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:45:50.970826 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 18:45:50.971139 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":331,"threads_started":1}
I20260811 18:45:50.973999 3775 heartbeater.cc:347] Connected to a master server at 127.30.185.190:43235
I20260811 18:45:50.974329 3775 heartbeater.cc:464] Registering TS with master...
I20260811 18:45:50.975106 3775 heartbeater.cc:511] Master 127.30.185.190:43235 requested a full tablet report, sending...
I20260811 18:45:50.977280 3442 ts_manager.cc:194] Registered new tserver with Master: ffa9ad378d684546b13506d6633927d1 (127.30.185.129:34779)
I20260811 18:45:50.978807 3442 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.185.129:51471
W20260811 18:45:51.318629 3782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:51.318930 3782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:51.319092 3782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:51.326525 3782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:51.326628 3782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:51.326807 3782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.130
I20260811 18:45:51.336251 3782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:35625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.185.130
--webserver_port=0
--tserver_master_addrs=127.30.185.190:43235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.130
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:51.337659 3782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:51.339457 3782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:51.352948 3789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:51.352949 3790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:51.353204 3793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:51.353646 3782 server_base.cc:1034] running on GCE node
I20260811 18:45:51.354242 3782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:51.355120 3782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:51.356360 3782 hybrid_clock.cc:648] HybridClock initialized: now 1786473951356309 us; error 56 us; skew 500 ppm
I20260811 18:45:51.356695 3782 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:45:51.358995 3782 webserver.cc:460] Webserver started at http://127.30.185.130:42177/ using document root <none> and password file <none>
I20260811 18:45:51.359715 3782 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:51.359853 3782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:51.360164 3782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:51.363085 3782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-1/data/instance:
uuid: "992ab69baa5648d3a791fdd657fd28e7"
format_stamp: "Formatted at 2026-08-11 18:45:51 on dist-test-slave-z8x9"
server_key: "c136f1c85c3e87c4381a3aaf7f348f43"
server_key_iv: "41d2f0b8c2e20185eafb8d786f50b0c9"
server_key_version: "encryptionkey@0"
I20260811 18:45:51.363898 3782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance:
uuid: "992ab69baa5648d3a791fdd657fd28e7"
format_stamp: "Formatted at 2026-08-11 18:45:51 on dist-test-slave-z8x9"
server_key: "c136f1c85c3e87c4381a3aaf7f348f43"
server_key_iv: "41d2f0b8c2e20185eafb8d786f50b0c9"
server_key_version: "encryptionkey@0"
I20260811 18:45:51.369148 3782 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 18:45:51.372787 3803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:51.374176 3782 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 18:45:51.374352 3782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-1/wal
uuid: "992ab69baa5648d3a791fdd657fd28e7"
format_stamp: "Formatted at 2026-08-11 18:45:51 on dist-test-slave-z8x9"
server_key: "c136f1c85c3e87c4381a3aaf7f348f43"
server_key_iv: "41d2f0b8c2e20185eafb8d786f50b0c9"
server_key_version: "encryptionkey@0"
I20260811 18:45:51.374571 3782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:51.395474 3782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:51.396436 3782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:51.396811 3782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:51.397946 3782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:45:51.399650 3782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:45:51.399784 3782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:51.399923 3782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:45:51.400012 3782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:51.435042 3782 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.130:41299
I20260811 18:45:51.435111 4013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.130:41299 every 8 connection(s)
I20260811 18:45:51.436426 3782 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-1/data/info.pb
I20260811 18:45:51.439409 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 3782
I20260811 18:45:51.439610 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-1/wal/instance
I20260811 18:45:51.440058 31462 external_mini_cluster.cc:1428] Setting key eb1cdbe27614adee12301085551ea569
I20260811 18:45:51.443058 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.185.131:0
--local_ip_for_outbound_sockets=127.30.185.131
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:43235
--builtin_ntp_servers=127.30.185.148:35625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 18:45:51.450770 3777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:51.439201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.130:36797 (local address 127.30.185.190:43235)
0811 18:45:51.439444 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:51.439453 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:51.440961 (+ 1508us) server_negotiation.cc:407] Connection header received
0811 18:45:51.442792 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:51.442800 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:51.442893 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:51.443012 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:51.444337 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:51.445565 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:51.447213 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:51.448108 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:51.448263 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:51.449790 (+ 1527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:51.449807 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:51.449815 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:51.449846 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:51.449890 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:51.449909 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:51.450003 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:51.450222 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 18:45:51.450363 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 18:45:51.451710 4017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:51.439705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:43235 (local address 127.30.185.130:36797)
0811 18:45:51.440777 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:51.440823 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:51.442520 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:51.443398 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:51.443408 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:51.443500 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:45:51.444118 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:51.444136 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:51.445720 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:51.445725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:51.447049 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:51.447063 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:51.448373 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:51.448378 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:51.448604 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:51.449433 (+ 829us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:45:51.449436 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:45:51.449480 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:45:51.449572 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:45:51.450016 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:45:51.450024 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:45:51.450031 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:45:51.450183 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 18:45:51.450517 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":338,"threads_started":1}
I20260811 18:45:51.453322 4014 heartbeater.cc:347] Connected to a master server at 127.30.185.190:43235
I20260811 18:45:51.453681 4014 heartbeater.cc:464] Registering TS with master...
I20260811 18:45:51.454697 4014 heartbeater.cc:511] Master 127.30.185.190:43235 requested a full tablet report, sending...
I20260811 18:45:51.456492 3442 ts_manager.cc:194] Registered new tserver with Master: 992ab69baa5648d3a791fdd657fd28e7 (127.30.185.130:41299)
I20260811 18:45:51.457435 3442 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.185.130:36797
W20260811 18:45:51.789288 4024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:51.789570 4024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:51.789656 4024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:51.796841 4024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:51.796952 4024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:51.797060 4024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.185.131
I20260811 18:45:51.806946 4024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:35625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.185.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.185.131
--webserver_port=0
--tserver_master_addrs=127.30.185.190:43235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.185.131
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:51.808276 4024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:51.810104 4024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:51.823144 4036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:51.823171 4042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:51.823246 4039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:51.824190 4024 server_base.cc:1034] running on GCE node
I20260811 18:45:51.824791 4024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:51.825827 4024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:51.827060 4024 hybrid_clock.cc:648] HybridClock initialized: now 1786473951827027 us; error 55 us; skew 500 ppm
I20260811 18:45:51.827382 4024 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 18:45:51.829679 4024 webserver.cc:460] Webserver started at http://127.30.185.131:42121/ using document root <none> and password file <none>
I20260811 18:45:51.830405 4024 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:51.830538 4024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:51.830858 4024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:51.833936 4024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-2/data/instance:
uuid: "d135aaf3239847009c5521148ad97b6d"
format_stamp: "Formatted at 2026-08-11 18:45:51 on dist-test-slave-z8x9"
server_key: "0016459d89fd23a75d9089a93883a166"
server_key_iv: "31998f780b220cfacc7a2db1c4d5b5d6"
server_key_version: "encryptionkey@0"
I20260811 18:45:51.834720 4024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance:
uuid: "d135aaf3239847009c5521148ad97b6d"
format_stamp: "Formatted at 2026-08-11 18:45:51 on dist-test-slave-z8x9"
server_key: "0016459d89fd23a75d9089a93883a166"
server_key_iv: "31998f780b220cfacc7a2db1c4d5b5d6"
server_key_version: "encryptionkey@0"
I20260811 18:45:51.839875 4024 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:45:51.843374 4052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:51.844695 4024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:45:51.844887 4024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-2/wal
uuid: "d135aaf3239847009c5521148ad97b6d"
format_stamp: "Formatted at 2026-08-11 18:45:51 on dist-test-slave-z8x9"
server_key: "0016459d89fd23a75d9089a93883a166"
server_key_iv: "31998f780b220cfacc7a2db1c4d5b5d6"
server_key_version: "encryptionkey@0"
I20260811 18:45:51.845114 4024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:51.870662 4024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:51.871618 4024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:51.871991 4024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:51.873059 4024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:45:51.874708 4024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:45:51.874847 4024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:51.874992 4024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:45:51.875088 4024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:51.912077 4024 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.131:33395
I20260811 18:45:51.912168 4267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.131:33395 every 8 connection(s)
I20260811 18:45:51.913447 4024 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-2/data/info.pb
I20260811 18:45:51.919111 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 4024
I20260811 18:45:51.919306 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-2/wal/instance
I20260811 18:45:51.919734 31462 external_mini_cluster.cc:1428] Setting key 2a3c6fb7a3d7098d77baa38312a98b4c
I20260811 18:45:51.926167 3777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:51.916038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.131:37497 (local address 127.30.185.190:43235)
0811 18:45:51.916203 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:51.916208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:51.917618 (+ 1410us) server_negotiation.cc:407] Connection header received
0811 18:45:51.919430 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:51.919437 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:51.919530 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:51.919639 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:51.920836 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:51.921848 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:51.923399 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:51.923800 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:51.923933 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:51.925338 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:51.925348 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:51.925351 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:51.925369 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:51.925400 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:51.925411 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:51.925504 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:51.925730 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 18:45:51.925834 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:45:51.927125 4271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:51.916516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:43235 (local address 127.30.185.131:37497)
0811 18:45:51.917460 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:51.917507 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:51.919136 (+ 1629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:51.919900 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:51.919909 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:51.919991 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:45:51.920660 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:51.920677 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:51.922000 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:51.922005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:51.923249 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:51.923261 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:51.924032 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:51.924041 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:51.924263 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:51.924974 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:45:51.924977 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:45:51.925016 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:45:51.925094 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:45:51.925507 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:45:51.925520 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:45:51.925529 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 18:45:51.925703 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 18:45:51.926098 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":279,"threads_started":1}
I20260811 18:45:51.928359 4268 heartbeater.cc:347] Connected to a master server at 127.30.185.190:43235
I20260811 18:45:51.928640 4268 heartbeater.cc:464] Registering TS with master...
I20260811 18:45:51.929426 4268 heartbeater.cc:511] Master 127.30.185.190:43235 requested a full tablet report, sending...
I20260811 18:45:51.930773 3442 ts_manager.cc:194] Registered new tserver with Master: d135aaf3239847009c5521148ad97b6d (127.30.185.131:33395)
I20260811 18:45:51.931591 3442 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.185.131:37497
I20260811 18:45:51.934731 31462 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:45:51.944778 3777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:51.939211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49658 (local address 127.30.185.190:43235)
0811 18:45:51.939453 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:51.939460 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:51.939866 (+ 406us) server_negotiation.cc:407] Connection header received
0811 18:45:51.940003 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:51.940007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:51.940081 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:51.940163 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:51.940621 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:51.941374 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:51.942652 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:51.943047 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:51.943137 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:51.944115 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:51.944120 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:51.944123 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:51.944140 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:51.944170 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:51.944181 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:51.944249 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:51.944374 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 18:45:51.944446 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 18:45:51.948376 3442 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:45:51.955897 3442 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:45:51.981973 3775 heartbeater.cc:503] Master 127.30.185.190:43235 was elected leader, sending a full tablet report...
I20260811 18:45:51.997548 4295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:51.969085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.129:34779 (local address 127.0.0.1:43578)
0811 18:45:51.969977 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:51.970025 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:51.970210 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:51.988844 (+ 18634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:51.988858 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:51.988922 (+ 64us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:51.989493 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:51.989512 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:51.991825 (+ 2313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:51.991831 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:51.994314 (+ 2483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:51.994330 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:51.996368 (+ 2038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:51.996373 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:51.996644 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:51.996705 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 18:45:51.996848 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 18:45:51.997052 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"mutex_wait_us":19,"thread_start_us":370,"threads_started":1}
I20260811 18:45:51.997892 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:51.969338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43578 (local address 127.30.185.129:34779)
0811 18:45:51.987664 (+ 18326us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:51.987678 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:51.987735 (+ 57us) server_negotiation.cc:407] Connection header received
0811 18:45:51.987879 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:51.987890 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:51.988234 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:51.989748 (+ 1514us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:51.990035 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:51.991643 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:51.994528 (+ 2885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:51.996101 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:51.996937 (+ 836us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:51.997192 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:51.997284 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 18:45:51.997513 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18085,"thread_start_us":19113,"threads_started":1}
I20260811 18:45:52.001435 4304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:51.970541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:41299 (local address 127.0.0.1:44410)
0811 18:45:51.992572 (+ 22031us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:51.992597 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:51.992811 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:51.993742 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:51.993746 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:51.993773 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:51.994553 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:51.994567 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:51.996139 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:51.996144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:51.998617 (+ 2473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:51.998634 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:52.000532 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:52.000538 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:52.000745 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:52.000778 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 18:45:52.000883 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 18:45:52.001018 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21890,"mutex_wait_us":21,"thread_start_us":21067,"threads_started":1}
I20260811 18:45:52.001606 4307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:51.971166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44410 (local address 127.30.185.130:41299)
0811 18:45:51.990077 (+ 18911us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:51.990087 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:51.992702 (+ 2615us) server_negotiation.cc:407] Connection header received
0811 18:45:51.992996 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:51.993003 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:51.993264 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:51.993472 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:51.994776 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:51.995746 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:51.998847 (+ 3101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:51.999988 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:52.000169 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:52.000388 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:52.001024 (+ 636us) server_negotiation.cc:299] Negotiation successful
0811 18:45:52.001226 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18732,"thread_start_us":18932,"threads_started":1}
I20260811 18:45:52.002393 4305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:51.969601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:33395 (local address 127.0.0.1:50482)
0811 18:45:51.990718 (+ 21117us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:51.990739 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:51.990856 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:51.993757 (+ 2901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:51.993760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:51.993832 (+ 72us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:51.994435 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:51.994451 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:51.996366 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:51.996371 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:51.999221 (+ 2850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:51.999237 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:52.001475 (+ 2238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:52.001481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:52.001646 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:52.001679 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 18:45:52.001898 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 18:45:52.002028 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20971,"mutex_wait_us":885,"spinlock_wait_cycles":1280,"thread_start_us":20657,"threads_started":1}
I20260811 18:45:52.002576 3666 tablet_service.cc:1467] Processing CreateTablet for tablet b3702eaa1cbe40faa13fec36e347848b (DEFAULT_TABLE table=test-table [id=2498d3e0f3114624840dff366f651b46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:45:52.003342 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:51.970080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50482 (local address 127.30.185.131:33395)
0811 18:45:51.992788 (+ 22708us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:51.992795 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:51.992823 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:45:51.992940 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:51.992948 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:51.993249 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:51.993472 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:51.994668 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:51.996209 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:51.999471 (+ 3262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:52.000905 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:52.001104 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:52.001325 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:52.002765 (+ 1440us) server_negotiation.cc:299] Negotiation successful
0811 18:45:52.002959 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22542,"thread_start_us":22380,"threads_started":1}
I20260811 18:45:52.004204 3666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3702eaa1cbe40faa13fec36e347848b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:52.005065 3887 tablet_service.cc:1467] Processing CreateTablet for tablet b3702eaa1cbe40faa13fec36e347848b (DEFAULT_TABLE table=test-table [id=2498d3e0f3114624840dff366f651b46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:45:52.006407 3887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3702eaa1cbe40faa13fec36e347848b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:52.012920 4137 tablet_service.cc:1467] Processing CreateTablet for tablet b3702eaa1cbe40faa13fec36e347848b (DEFAULT_TABLE table=test-table [id=2498d3e0f3114624840dff366f651b46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:45:52.014566 4137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3702eaa1cbe40faa13fec36e347848b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:52.018601 4311 tablet_bootstrap.cc:492] T b3702eaa1cbe40faa13fec36e347848b P 992ab69baa5648d3a791fdd657fd28e7: Bootstrap starting.
I20260811 18:45:52.020704 4312 tablet_bootstrap.cc:492] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1: Bootstrap starting.
I20260811 18:45:52.022665 4311 tablet_bootstrap.cc:654] T b3702eaa1cbe40faa13fec36e347848b P 992ab69baa5648d3a791fdd657fd28e7: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:52.024029 4311 log.cc:824] T b3702eaa1cbe40faa13fec36e347848b P 992ab69baa5648d3a791fdd657fd28e7: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:52.024451 4312 tablet_bootstrap.cc:654] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:52.026026 4312 log.cc:824] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:52.028451 4316 tablet_bootstrap.cc:492] T b3702eaa1cbe40faa13fec36e347848b P d135aaf3239847009c5521148ad97b6d: Bootstrap starting.
I20260811 18:45:52.029150 4311 tablet_bootstrap.cc:492] T b3702eaa1cbe40faa13fec36e347848b P 992ab69baa5648d3a791fdd657fd28e7: No bootstrap required, opened a new log
I20260811 18:45:52.029247 4312 tablet_bootstrap.cc:492] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1: No bootstrap required, opened a new log
I20260811 18:45:52.029467 4311 ts_tablet_manager.cc:1397] T b3702eaa1cbe40faa13fec36e347848b P 992ab69baa5648d3a791fdd657fd28e7: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260811 18:45:52.029502 4312 ts_tablet_manager.cc:1397] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 18:45:52.032233 4316 tablet_bootstrap.cc:654] T b3702eaa1cbe40faa13fec36e347848b P d135aaf3239847009c5521148ad97b6d: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:52.034005 4316 log.cc:824] T b3702eaa1cbe40faa13fec36e347848b P d135aaf3239847009c5521148ad97b6d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:52.037417 4316 tablet_bootstrap.cc:492] T b3702eaa1cbe40faa13fec36e347848b P d135aaf3239847009c5521148ad97b6d: No bootstrap required, opened a new log
I20260811 18:45:52.037714 4316 ts_tablet_manager.cc:1397] T b3702eaa1cbe40faa13fec36e347848b P d135aaf3239847009c5521148ad97b6d: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 18:45:52.037739 4312 raft_consensus.cc:348] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffa9ad378d684546b13506d6633927d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34779 } } peers { permanent_uuid: "992ab69baa5648d3a791fdd657fd28e7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 41299 } } peers { permanent_uuid: "d135aaf3239847009c5521148ad97b6d" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 33395 } }
I20260811 18:45:52.038144 4312 raft_consensus.cc:374] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:52.038264 4312 raft_consensus.cc:729] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffa9ad378d684546b13506d6633927d1, State: Initialized, Role: FOLLOWER
I20260811 18:45:52.038863 4312 consensus_queue.cc:260] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffa9ad378d684546b13506d6633927d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34779 } } peers { permanent_uuid: "992ab69baa5648d3a791fdd657fd28e7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 41299 } } peers { permanent_uuid: "d135aaf3239847009c5521148ad97b6d" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 33395 } }
I20260811 18:45:52.039901 4311 raft_consensus.cc:348] T b3702eaa1cbe40faa13fec36e347848b P 992ab69baa5648d3a791fdd657fd28e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffa9ad378d684546b13506d6633927d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34779 } } peers { permanent_uuid: "992ab69baa5648d3a791fdd657fd28e7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 41299 } } peers { permanent_uuid: "d135aaf3239847009c5521148ad97b6d" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 33395 } }
I20260811 18:45:52.040436 4311 raft_consensus.cc:374] T b3702eaa1cbe40faa13fec36e347848b P 992ab69baa5648d3a791fdd657fd28e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:52.040539 4312 ts_tablet_manager.cc:1428] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260811 18:45:52.040544 4311 raft_consensus.cc:729] T b3702eaa1cbe40faa13fec36e347848b P 992ab69baa5648d3a791fdd657fd28e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 992ab69baa5648d3a791fdd657fd28e7, State: Initialized, Role: FOLLOWER
I20260811 18:45:52.041144 4311 consensus_queue.cc:260] T b3702eaa1cbe40faa13fec36e347848b P 992ab69baa5648d3a791fdd657fd28e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffa9ad378d684546b13506d6633927d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34779 } } peers { permanent_uuid: "992ab69baa5648d3a791fdd657fd28e7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 41299 } } peers { permanent_uuid: "d135aaf3239847009c5521148ad97b6d" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 33395 } }
I20260811 18:45:52.044291 4014 heartbeater.cc:503] Master 127.30.185.190:43235 was elected leader, sending a full tablet report...
I20260811 18:45:52.045006 4311 ts_tablet_manager.cc:1428] T b3702eaa1cbe40faa13fec36e347848b P 992ab69baa5648d3a791fdd657fd28e7: Time spent starting tablet: real 0.015s user 0.006s sys 0.008s
I20260811 18:45:52.050601 4316 raft_consensus.cc:348] T b3702eaa1cbe40faa13fec36e347848b P d135aaf3239847009c5521148ad97b6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffa9ad378d684546b13506d6633927d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34779 } } peers { permanent_uuid: "992ab69baa5648d3a791fdd657fd28e7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 41299 } } peers { permanent_uuid: "d135aaf3239847009c5521148ad97b6d" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 33395 } }
I20260811 18:45:52.051061 4316 raft_consensus.cc:374] T b3702eaa1cbe40faa13fec36e347848b P d135aaf3239847009c5521148ad97b6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:52.051169 4316 raft_consensus.cc:729] T b3702eaa1cbe40faa13fec36e347848b P d135aaf3239847009c5521148ad97b6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d135aaf3239847009c5521148ad97b6d, State: Initialized, Role: FOLLOWER
I20260811 18:45:52.051741 4316 consensus_queue.cc:260] T b3702eaa1cbe40faa13fec36e347848b P d135aaf3239847009c5521148ad97b6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffa9ad378d684546b13506d6633927d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34779 } } peers { permanent_uuid: "992ab69baa5648d3a791fdd657fd28e7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 41299 } } peers { permanent_uuid: "d135aaf3239847009c5521148ad97b6d" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 33395 } }
I20260811 18:45:52.052713 4268 heartbeater.cc:503] Master 127.30.185.190:43235 was elected leader, sending a full tablet report...
I20260811 18:45:52.053287 4316 ts_tablet_manager.cc:1428] T b3702eaa1cbe40faa13fec36e347848b P d135aaf3239847009c5521148ad97b6d: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260811 18:45:52.070236 4319 raft_consensus.cc:482] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:45:52.070593 4319 raft_consensus.cc:504] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffa9ad378d684546b13506d6633927d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34779 } } peers { permanent_uuid: "992ab69baa5648d3a791fdd657fd28e7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 41299 } } peers { permanent_uuid: "d135aaf3239847009c5521148ad97b6d" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 33395 } }
I20260811 18:45:52.072415 4319 leader_election.cc:288] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 992ab69baa5648d3a791fdd657fd28e7 (127.30.185.130:41299), d135aaf3239847009c5521148ad97b6d (127.30.185.131:33395)
I20260811 18:45:52.080575 4307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:52.072962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:37961 (local address 127.30.185.130:41299)
0811 18:45:52.073247 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:52.073255 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:52.074329 (+ 1074us) server_negotiation.cc:407] Connection header received
0811 18:45:52.074500 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:52.074506 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:52.074608 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:52.074738 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:52.075464 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:52.076346 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:52.078345 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:52.079556 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:52.079724 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:52.079799 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:52.080176 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 18:45:52.080298 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 18:45:52.080763 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:52.073093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.185.129:36229 (local address 127.30.185.131:33395)
0811 18:45:52.073372 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:52.073381 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:52.073947 (+ 566us) server_negotiation.cc:407] Connection header received
0811 18:45:52.074280 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:52.074288 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:52.074366 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:52.074458 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:45:52.075207 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:52.076019 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:52.078546 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:52.079806 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:52.079938 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:52.079983 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:52.080363 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 18:45:52.080490 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 18:45:52.080762 4322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:52.072829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.130:41299 (local address 127.30.185.129:37961)
0811 18:45:52.074207 (+ 1378us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:52.074224 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:52.074351 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:52.074750 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:52.074756 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:52.074782 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:52.075263 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:52.075277 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:52.076522 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:52.076527 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:52.078139 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:52.078152 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:52.079768 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:52.079774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:52.079969 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:52.080017 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 18:45:52.080138 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 18:45:52.080296 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1257,"mutex_wait_us":38,"thread_start_us":610,"threads_started":1}
I20260811 18:45:52.080786 4325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:52.072666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.131:33395 (local address 127.30.185.129:36229)
0811 18:45:52.073827 (+ 1161us) negotiation.cc:107] Waiting for socket to connect
0811 18:45:52.073849 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:45:52.074093 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:45:52.074497 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:45:52.074503 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:45:52.074544 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:45:52.075053 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:52.075066 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:52.076191 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:52.076198 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:52.078357 (+ 2159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:45:52.078371 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:45:52.080008 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:45:52.080014 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:45:52.080198 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:52.080231 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 18:45:52.080315 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 18:45:52.080430 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1004,"mutex_wait_us":26,"thread_start_us":1227,"threads_started":1}
I20260811 18:45:52.081955 3930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3702eaa1cbe40faa13fec36e347848b" candidate_uuid: "ffa9ad378d684546b13506d6633927d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "992ab69baa5648d3a791fdd657fd28e7" is_pre_election: true
I20260811 18:45:52.081986 4178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3702eaa1cbe40faa13fec36e347848b" candidate_uuid: "ffa9ad378d684546b13506d6633927d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d135aaf3239847009c5521148ad97b6d" is_pre_election: true
I20260811 18:45:52.082459 3930 raft_consensus.cc:2454] T b3702eaa1cbe40faa13fec36e347848b P 992ab69baa5648d3a791fdd657fd28e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffa9ad378d684546b13506d6633927d1 in term 0.
I20260811 18:45:52.082502 4178 raft_consensus.cc:2454] T b3702eaa1cbe40faa13fec36e347848b P d135aaf3239847009c5521148ad97b6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffa9ad378d684546b13506d6633927d1 in term 0.
I20260811 18:45:52.083194 3584 leader_election.cc:302] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 992ab69baa5648d3a791fdd657fd28e7, d135aaf3239847009c5521148ad97b6d, ffa9ad378d684546b13506d6633927d1; no voters:
I20260811 18:45:52.083595 4319 raft_consensus.cc:2781] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:45:52.083771 4319 raft_consensus.cc:482] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:45:52.083894 4319 raft_consensus.cc:3037] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:52.086391 4319 raft_consensus.cc:504] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffa9ad378d684546b13506d6633927d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34779 } } peers { permanent_uuid: "992ab69baa5648d3a791fdd657fd28e7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 41299 } } peers { permanent_uuid: "d135aaf3239847009c5521148ad97b6d" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 33395 } }
I20260811 18:45:52.087096 4319 leader_election.cc:288] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1 [CANDIDATE]: Term 1 election: Requested vote from peers 992ab69baa5648d3a791fdd657fd28e7 (127.30.185.130:41299), d135aaf3239847009c5521148ad97b6d (127.30.185.131:33395)
I20260811 18:45:52.087589 3930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3702eaa1cbe40faa13fec36e347848b" candidate_uuid: "ffa9ad378d684546b13506d6633927d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "992ab69baa5648d3a791fdd657fd28e7"
I20260811 18:45:52.087590 4178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3702eaa1cbe40faa13fec36e347848b" candidate_uuid: "ffa9ad378d684546b13506d6633927d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d135aaf3239847009c5521148ad97b6d"
I20260811 18:45:52.087806 3930 raft_consensus.cc:3037] T b3702eaa1cbe40faa13fec36e347848b P 992ab69baa5648d3a791fdd657fd28e7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:52.087815 4178 raft_consensus.cc:3037] T b3702eaa1cbe40faa13fec36e347848b P d135aaf3239847009c5521148ad97b6d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:52.090148 3930 raft_consensus.cc:2454] T b3702eaa1cbe40faa13fec36e347848b P 992ab69baa5648d3a791fdd657fd28e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffa9ad378d684546b13506d6633927d1 in term 1.
I20260811 18:45:52.090296 4178 raft_consensus.cc:2454] T b3702eaa1cbe40faa13fec36e347848b P d135aaf3239847009c5521148ad97b6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffa9ad378d684546b13506d6633927d1 in term 1.
I20260811 18:45:52.090627 3584 leader_election.cc:302] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 992ab69baa5648d3a791fdd657fd28e7, ffa9ad378d684546b13506d6633927d1; no voters:
I20260811 18:45:52.090948 4319 raft_consensus.cc:2781] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:45:52.091423 4319 raft_consensus.cc:686] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1 [term 1 LEADER]: Becoming Leader. State: Replica: ffa9ad378d684546b13506d6633927d1, State: Running, Role: LEADER
I20260811 18:45:52.091934 4319 consensus_queue.cc:237] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffa9ad378d684546b13506d6633927d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34779 } } peers { permanent_uuid: "992ab69baa5648d3a791fdd657fd28e7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 41299 } } peers { permanent_uuid: "d135aaf3239847009c5521148ad97b6d" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 33395 } }
I20260811 18:45:52.097496 3442 catalog_manager.cc:5495] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffa9ad378d684546b13506d6633927d1 (127.30.185.129). New cstate: current_term: 1 leader_uuid: "ffa9ad378d684546b13506d6633927d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffa9ad378d684546b13506d6633927d1" member_type: VOTER last_known_addr { host: "127.30.185.129" port: 34779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "992ab69baa5648d3a791fdd657fd28e7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 41299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d135aaf3239847009c5521148ad97b6d" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 33395 } health_report { overall_health: UNKNOWN } } }
I20260811 18:45:52.130987 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:52.125348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43582 (local address 127.30.185.129:34779)
0811 18:45:52.125576 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:52.125583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:52.125603 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:45:52.125861 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:52.125865 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:52.125976 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:52.126141 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:45:52.126531 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:52.127259 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:52.128710 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:52.129130 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:52.129333 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:52.129859 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:45:52.130405 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:45:52.130571 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:52.130630 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 18:45:52.130717 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 18:45:52.139829 3930 raft_consensus.cc:1261] T b3702eaa1cbe40faa13fec36e347848b P 992ab69baa5648d3a791fdd657fd28e7 [term 1 FOLLOWER]: Refusing update from remote peer ffa9ad378d684546b13506d6633927d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:45:52.139848 4178 raft_consensus.cc:1261] T b3702eaa1cbe40faa13fec36e347848b P d135aaf3239847009c5521148ad97b6d [term 1 FOLLOWER]: Refusing update from remote peer ffa9ad378d684546b13506d6633927d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:45:52.140977 4326 consensus_queue.cc:1035] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d135aaf3239847009c5521148ad97b6d" member_type: VOTER last_known_addr { host: "127.30.185.131" port: 33395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:45:52.141470 4319 consensus_queue.cc:1035] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "992ab69baa5648d3a791fdd657fd28e7" member_type: VOTER last_known_addr { host: "127.30.185.130" port: 41299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:45:52.155676 4329 mvcc.cc:205] Tried to move back new op lower bound from 7317397307953037312 to 7317397307773259776. Current Snapshot: MvccSnapshot[applied={T|T < 7317397307953037312}]
I20260811 18:45:52.169596 3442 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:45:52.169919 3442 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:45:52.174201 3442 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42dbcacc80734b0c8b502cc0dccd05a5: Sending DeleteTablet for 3 replicas of tablet b3702eaa1cbe40faa13fec36e347848b
I20260811 18:45:52.175514 3666 tablet_service.cc:1514] Processing DeleteTablet for tablet b3702eaa1cbe40faa13fec36e347848b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:45:52 UTC) from {username='slave'} at 127.0.0.1:43578
I20260811 18:45:52.175653 4137 tablet_service.cc:1514] Processing DeleteTablet for tablet b3702eaa1cbe40faa13fec36e347848b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:45:52 UTC) from {username='slave'} at 127.0.0.1:50482
I20260811 18:45:52.175922 3887 tablet_service.cc:1514] Processing DeleteTablet for tablet b3702eaa1cbe40faa13fec36e347848b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:45:52 UTC) from {username='slave'} at 127.0.0.1:44410
I20260811 18:45:52.183282 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 3558
I20260811 18:45:52.186929 4349 tablet_replica.cc:330] T b3702eaa1cbe40faa13fec36e347848b P 992ab69baa5648d3a791fdd657fd28e7: stopping tablet replica
I20260811 18:45:52.187727 4349 raft_consensus.cc:2229] T b3702eaa1cbe40faa13fec36e347848b P 992ab69baa5648d3a791fdd657fd28e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:45:52.188519 4349 raft_consensus.cc:2258] T b3702eaa1cbe40faa13fec36e347848b P 992ab69baa5648d3a791fdd657fd28e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:45:52.189581 4347 tablet_replica.cc:330] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1: stopping tablet replica
I20260811 18:45:52.190191 4347 raft_consensus.cc:2229] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:45:52.191078 4347 raft_consensus.cc:2258] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:45:52.193975 4347 ts_tablet_manager.cc:1905] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:45:52.194327 4349 ts_tablet_manager.cc:1905] T b3702eaa1cbe40faa13fec36e347848b P 992ab69baa5648d3a791fdd657fd28e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:45:52.194645 4348 tablet_replica.cc:330] T b3702eaa1cbe40faa13fec36e347848b P d135aaf3239847009c5521148ad97b6d: stopping tablet replica
I20260811 18:45:52.195225 4348 raft_consensus.cc:2229] T b3702eaa1cbe40faa13fec36e347848b P d135aaf3239847009c5521148ad97b6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:45:52.195760 4348 raft_consensus.cc:2258] T b3702eaa1cbe40faa13fec36e347848b P d135aaf3239847009c5521148ad97b6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:45:52.205060 4349 ts_tablet_manager.cc:1918] T b3702eaa1cbe40faa13fec36e347848b P 992ab69baa5648d3a791fdd657fd28e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:45:52.205374 4349 log.cc:1196] T b3702eaa1cbe40faa13fec36e347848b P 992ab69baa5648d3a791fdd657fd28e7: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-1/wal/wals/b3702eaa1cbe40faa13fec36e347848b
I20260811 18:45:52.205757 4348 ts_tablet_manager.cc:1905] T b3702eaa1cbe40faa13fec36e347848b P d135aaf3239847009c5521148ad97b6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:45:52.206256 4349 ts_tablet_manager.cc:1939] T b3702eaa1cbe40faa13fec36e347848b P 992ab69baa5648d3a791fdd657fd28e7: Deleting consensus metadata
I20260811 18:45:52.207334 4347 ts_tablet_manager.cc:1918] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:45:52.207523 4347 log.cc:1196] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-0/wal/wals/b3702eaa1cbe40faa13fec36e347848b
I20260811 18:45:52.208127 4347 ts_tablet_manager.cc:1939] T b3702eaa1cbe40faa13fec36e347848b P ffa9ad378d684546b13506d6633927d1: Deleting consensus metadata
I20260811 18:45:52.209018 3418 catalog_manager.cc:4841] TS 992ab69baa5648d3a791fdd657fd28e7 (127.30.185.130:41299): tablet b3702eaa1cbe40faa13fec36e347848b (table test-table [id=2498d3e0f3114624840dff366f651b46]) successfully deleted
I20260811 18:45:52.209829 3418 catalog_manager.cc:4841] TS ffa9ad378d684546b13506d6633927d1 (127.30.185.129:34779): tablet b3702eaa1cbe40faa13fec36e347848b (table test-table [id=2498d3e0f3114624840dff366f651b46]) successfully deleted
I20260811 18:45:52.212442 4348 ts_tablet_manager.cc:1918] T b3702eaa1cbe40faa13fec36e347848b P d135aaf3239847009c5521148ad97b6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:45:52.212615 4348 log.cc:1196] T b3702eaa1cbe40faa13fec36e347848b P d135aaf3239847009c5521148ad97b6d: Deleting WAL directory at /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786473610573616-31462-0/minicluster-data/ts-2/wal/wals/b3702eaa1cbe40faa13fec36e347848b
I20260811 18:45:52.213213 4348 ts_tablet_manager.cc:1939] T b3702eaa1cbe40faa13fec36e347848b P d135aaf3239847009c5521148ad97b6d: Deleting consensus metadata
I20260811 18:45:52.213620 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:52.184990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43594 (local address 127.30.185.129:34779)
0811 18:45:52.185342 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:52.185352 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:52.200041 (+ 14689us) server_negotiation.cc:407] Connection header received
0811 18:45:52.200285 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:52.200294 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:52.200406 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:52.200505 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:52.201336 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:52.202636 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:52.204713 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:52.205381 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:52.205495 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:52.211820 (+ 6325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:52.211833 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:52.211851 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:52.212651 (+ 800us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:52.212726 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:52.212751 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:52.212886 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:52.213078 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 18:45:52.213225 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260811 18:45:52.214411 3768 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:52.215142 3420 catalog_manager.cc:4841] TS d135aaf3239847009c5521148ad97b6d (127.30.185.131:33395): tablet b3702eaa1cbe40faa13fec36e347848b (table test-table [id=2498d3e0f3114624840dff366f651b46]) successfully deleted
I20260811 18:45:52.313413 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 3558
I20260811 18:45:52.337343 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 3782
I20260811 18:45:52.344599 4307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:52.338187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44424 (local address 127.30.185.130:41299)
0811 18:45:52.338380 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:52.338387 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:52.338463 (+ 76us) server_negotiation.cc:407] Connection header received
0811 18:45:52.338671 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:52.338677 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:52.338765 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:52.338888 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:52.339301 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:52.340059 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:52.341555 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:52.341997 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:52.342100 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:52.343247 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:52.343256 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:52.343278 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:52.343858 (+ 580us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:52.343905 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:52.343923 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:52.344017 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:52.344149 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 18:45:52.344239 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 18:45:52.345366 4006 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:52.451113 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 3782
I20260811 18:45:52.476015 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 4024
I20260811 18:45:52.483172 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:52.476937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39162 (local address 127.30.185.131:33395)
0811 18:45:52.477142 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:52.477148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:52.477228 (+ 80us) server_negotiation.cc:407] Connection header received
0811 18:45:52.477458 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:52.477464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:52.477563 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:52.477666 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:52.478120 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:52.478907 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:52.480182 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:52.480594 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:52.480691 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:52.481838 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:52.481847 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:52.481863 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:52.482428 (+ 565us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:52.482476 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:52.482496 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:52.482593 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:52.482718 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 18:45:52.482820 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 18:45:52.483882 4258 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:52.573161 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 4024
I20260811 18:45:52.602319 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 3388
I20260811 18:45:52.603461 3498 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:52.720712 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 3388
2026-08-11T18:45:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2921 ms)
[----------] 6 tests from EncryptionPolicyTest (24215 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:45:52 dist-test-slave-z8x9 krb5kdc[4362](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:45:52 dist-test-slave-z8x9 krb5kdc[4362](info): set up 2 sockets
Aug 11 18:45:52 dist-test-slave-z8x9 krb5kdc[4362](info): commencing operation
krb5kdc: starting...
W20260811 18:45:54.153296 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.373s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:45:54 dist-test-slave-z8x9 krb5kdc[4362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:45:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:45:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:45:54.509851 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37137
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:45:54.866611 4391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:54.866928 4391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:54.867026 4391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:54.874414 4391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:45:54.874542 4391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:54.874601 4391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:54.874671 4391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:45:54.874717 4391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:45:54.884958 4391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37137
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:54.886343 4391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:54.887616 4391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:54.899701 4408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:54.899777 4401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:54.899797 4399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:54.900633 4391 server_base.cc:1034] running on GCE node
I20260811 18:45:54.901542 4391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:54.903044 4391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:54.904284 4391 hybrid_clock.cc:648] HybridClock initialized: now 1786473954904212 us; error 61 us; skew 500 ppm
I20260811 18:45:54.904657 4391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:45:54 dist-test-slave-z8x9 krb5kdc[4362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473954, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:45:54.908115 4391 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:45:54.909760 4391 webserver.cc:460] Webserver started at http://127.0.0.1:38347/ using document root <none> and password file <none>
I20260811 18:45:54.910457 4391 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:54.910593 4391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:54.910909 4391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:54.914201 4391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "5b9b8d91f8704a4f8a06ef429790dd76"
format_stamp: "Formatted at 2026-08-11 18:45:54 on dist-test-slave-z8x9"
server_key: "7d4dc1e52c790e8ea664692ed5e5f76c"
server_key_iv: "90bfddb50174fa9d4240c543e2728fc5"
server_key_version: "encryptionkey@0"
I20260811 18:45:54.915007 4391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "5b9b8d91f8704a4f8a06ef429790dd76"
format_stamp: "Formatted at 2026-08-11 18:45:54 on dist-test-slave-z8x9"
server_key: "7d4dc1e52c790e8ea664692ed5e5f76c"
server_key_iv: "90bfddb50174fa9d4240c543e2728fc5"
server_key_version: "encryptionkey@0"
I20260811 18:45:54.920421 4391 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:45:54.924031 4417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:54.925356 4391 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:45:54.925545 4391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "5b9b8d91f8704a4f8a06ef429790dd76"
format_stamp: "Formatted at 2026-08-11 18:45:54 on dist-test-slave-z8x9"
server_key: "7d4dc1e52c790e8ea664692ed5e5f76c"
server_key_iv: "90bfddb50174fa9d4240c543e2728fc5"
server_key_version: "encryptionkey@0"
I20260811 18:45:54.925750 4391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:54.966251 4391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:54.971119 4391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:54.971457 4391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:54.991175 4391 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37137
I20260811 18:45:54.991200 4510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37137 every 8 connection(s)
I20260811 18:45:54.992704 4391 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:45:54.995086 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 4391
I20260811 18:45:54.995334 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:45:54.996199 31462 external_mini_cluster.cc:1428] Setting key 5767ebcf065324a48c4e4304ffcfdd46
I20260811 18:45:54.997272 4511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 18:45:55 dist-test-slave-z8x9 krb5kdc[4362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 18:45:55.012573 4511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b9b8d91f8704a4f8a06ef429790dd76: Bootstrap starting.
I20260811 18:45:55.015938 4511 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b9b8d91f8704a4f8a06ef429790dd76: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:55.017298 4511 log.cc:824] T 00000000000000000000000000000000 P 5b9b8d91f8704a4f8a06ef429790dd76: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:55.020794 4511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b9b8d91f8704a4f8a06ef429790dd76: No bootstrap required, opened a new log
I20260811 18:45:55.021276 4518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:54.998287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49567 (local address 127.0.0.1:37137)
0811 18:45:54.999112 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:54.999124 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:54.999167 (+ 43us) server_negotiation.cc:407] Connection header received
0811 18:45:55.000703 (+ 1536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:55.000725 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:55.001128 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:55.001569 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:55.003415 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:55.004870 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:55.006790 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:55.007473 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:55.007602 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:55.012970 (+ 5368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:55.012978 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:55.012993 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:55.013037 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:55.016355 (+ 3318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:55.017451 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:55.017462 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:55.017471 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:55.017647 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:55.018228 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:55.018235 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:55.018241 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:55.018960 (+ 719us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:55.019214 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:55.019775 (+ 561us) server_negotiation.cc:299] Negotiation successful
0811 18:45:55.020060 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":374,"threads_started":1}
I20260811 18:45:55.025686 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:45:55.031719 4511 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b9b8d91f8704a4f8a06ef429790dd76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b9b8d91f8704a4f8a06ef429790dd76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37137 } }
I20260811 18:45:55.032255 4511 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b9b8d91f8704a4f8a06ef429790dd76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:55.032375 4511 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b9b8d91f8704a4f8a06ef429790dd76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b9b8d91f8704a4f8a06ef429790dd76, State: Initialized, Role: FOLLOWER
I20260811 18:45:55.033116 4511 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b9b8d91f8704a4f8a06ef429790dd76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b9b8d91f8704a4f8a06ef429790dd76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37137 } }
I20260811 18:45:55.033433 4511 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b9b8d91f8704a4f8a06ef429790dd76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:45:55.033591 4511 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b9b8d91f8704a4f8a06ef429790dd76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:45:55.033782 4511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b9b8d91f8704a4f8a06ef429790dd76 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:55.036686 4511 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b9b8d91f8704a4f8a06ef429790dd76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b9b8d91f8704a4f8a06ef429790dd76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37137 } }
I20260811 18:45:55.037374 4511 leader_election.cc:302] T 00000000000000000000000000000000 P 5b9b8d91f8704a4f8a06ef429790dd76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b9b8d91f8704a4f8a06ef429790dd76; no voters:
I20260811 18:45:55.038102 4511 leader_election.cc:288] T 00000000000000000000000000000000 P 5b9b8d91f8704a4f8a06ef429790dd76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:45:55.038218 4540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b9b8d91f8704a4f8a06ef429790dd76 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:45:55.039451 4540 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b9b8d91f8704a4f8a06ef429790dd76 [term 1 LEADER]: Becoming Leader. State: Replica: 5b9b8d91f8704a4f8a06ef429790dd76, State: Running, Role: LEADER
I20260811 18:45:55.040123 4540 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b9b8d91f8704a4f8a06ef429790dd76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b9b8d91f8704a4f8a06ef429790dd76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37137 } }
I20260811 18:45:55.040434 4511 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b9b8d91f8704a4f8a06ef429790dd76 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:45:55.044454 4542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b9b8d91f8704a4f8a06ef429790dd76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b9b8d91f8704a4f8a06ef429790dd76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b9b8d91f8704a4f8a06ef429790dd76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37137 } } }
I20260811 18:45:55.044966 4542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b9b8d91f8704a4f8a06ef429790dd76 [sys.catalog]: This master's current role is: LEADER
I20260811 18:45:55.045521 4543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b9b8d91f8704a4f8a06ef429790dd76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b9b8d91f8704a4f8a06ef429790dd76. Latest consensus state: current_term: 1 leader_uuid: "5b9b8d91f8704a4f8a06ef429790dd76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b9b8d91f8704a4f8a06ef429790dd76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37137 } } }
I20260811 18:45:55.045874 4543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b9b8d91f8704a4f8a06ef429790dd76 [sys.catalog]: This master's current role is: LEADER
I20260811 18:45:55.052425 4518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:55.031857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45003 (local address 127.0.0.1:37137)
0811 18:45:55.032058 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:55.032069 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:55.032598 (+ 529us) server_negotiation.cc:407] Connection header received
0811 18:45:55.032762 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:55.032771 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:55.032877 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:55.032995 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:55.034662 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:55.035834 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:55.037116 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:55.037850 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:55.038013 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:55.041587 (+ 3574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:55.041594 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:55.041598 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:45:55.041643 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:55.046212 (+ 4569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:55.047664 (+ 1452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:55.047672 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:55.047678 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:55.047906 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:45:55.048808 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:45:55.048814 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:45:55.048817 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:45:55.049435 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:55.051369 (+ 1934us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:55.051606 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 18:45:55.051856 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 18:45:55.054371 4549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:45:55.059751 4549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:45:55.069231 4549 catalog_manager.cc:1347] Generated new cluster ID: 9b56d1db20cd4fe2b9e196d717d4f1ff
I20260811 18:45:55.069376 4549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:45:55.087687 4549 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:45:55.089046 4549 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:45:55.098106 4549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b9b8d91f8704a4f8a06ef429790dd76: Generated new TSK 0
I20260811 18:45:55.098946 4549 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:45:55.108848 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 4391
I20260811 18:45:55.110577 4504 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:45:55.203346 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 4391
W20260811 18:45:55.229931 4359 connection.cc:535] client connection to 127.0.0.1:37137 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37137): Connection reset by peer (error 104)
2026-08-11T18:45:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2471 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:45:55 dist-test-slave-z8x9 krb5kdc[4581](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:45:55 dist-test-slave-z8x9 krb5kdc[4581](info): set up 2 sockets
Aug 11 18:45:55 dist-test-slave-z8x9 krb5kdc[4581](info): commencing operation
krb5kdc: starting...
W20260811 18:45:56.632290 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.382s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:45:56 dist-test-slave-z8x9 krb5kdc[4581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:45:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:45:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:45:56.992177 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39127
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:45:57.345610 4606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:57.345924 4606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:57.346020 4606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:57.353549 4606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:45:57.353670 4606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:57.353718 4606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:57.353783 4606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:45:57.353829 4606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:45:57.363955 4606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39127
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:57.365342 4606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:57.366654 4606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:57.379376 4618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:57.379452 4615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:57.380036 4614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:57.380110 4606 server_base.cc:1034] running on GCE node
I20260811 18:45:57.381038 4606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:57.382689 4606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:57.383936 4606 hybrid_clock.cc:648] HybridClock initialized: now 1786473957383889 us; error 54 us; skew 500 ppm
I20260811 18:45:57.384332 4606 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:45:57 dist-test-slave-z8x9 krb5kdc[4581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473957, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:45:57.393427 4606 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:45:57.395191 4606 webserver.cc:460] Webserver started at http://127.0.0.1:36299/ using document root <none> and password file <none>
I20260811 18:45:57.396009 4606 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:57.396183 4606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:57.396538 4606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:57.400059 4606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "4ac1dca992594315a09eeba53f9b401b"
format_stamp: "Formatted at 2026-08-11 18:45:57 on dist-test-slave-z8x9"
server_key: "eb27b2992080e4ee8ad01e9c0a7f9c6d"
server_key_iv: "391ba18a574e6b3839c2a0e0eca117df"
server_key_version: "encryptionkey@0"
I20260811 18:45:57.400977 4606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "4ac1dca992594315a09eeba53f9b401b"
format_stamp: "Formatted at 2026-08-11 18:45:57 on dist-test-slave-z8x9"
server_key: "eb27b2992080e4ee8ad01e9c0a7f9c6d"
server_key_iv: "391ba18a574e6b3839c2a0e0eca117df"
server_key_version: "encryptionkey@0"
I20260811 18:45:57.406930 4606 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 18:45:57.410821 4633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:57.412242 4606 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:45:57.412451 4606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "4ac1dca992594315a09eeba53f9b401b"
format_stamp: "Formatted at 2026-08-11 18:45:57 on dist-test-slave-z8x9"
server_key: "eb27b2992080e4ee8ad01e9c0a7f9c6d"
server_key_iv: "391ba18a574e6b3839c2a0e0eca117df"
server_key_version: "encryptionkey@0"
I20260811 18:45:57.412674 4606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:57.428861 4606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:57.433920 4606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:57.434253 4606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:57.455041 4606 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39127
I20260811 18:45:57.455065 4738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39127 every 8 connection(s)
I20260811 18:45:57.456655 4606 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:45:57.460426 4739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:57.466866 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 4606
I20260811 18:45:57.467087 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:45:57.467558 31462 external_mini_cluster.cc:1428] Setting key c10d98b30aaacec4a0fa34b62055b647
I20260811 18:45:57.471252 4739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ac1dca992594315a09eeba53f9b401b: Bootstrap starting.
I20260811 18:45:57.474776 4739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ac1dca992594315a09eeba53f9b401b: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:57.476130 4739 log.cc:824] T 00000000000000000000000000000000 P 4ac1dca992594315a09eeba53f9b401b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:57.479610 4739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ac1dca992594315a09eeba53f9b401b: No bootstrap required, opened a new log
I20260811 18:45:57.481952 4745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:57.469495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46061 (local address 127.0.0.1:39127)
0811 18:45:57.470310 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:57.470322 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:57.470366 (+ 44us) server_negotiation.cc:407] Connection header received
0811 18:45:57.472195 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:57.472219 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:57.472625 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:57.472973 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:57.473702 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:57.475062 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:57.477034 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:57.477975 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:57.478108 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:57.479481 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:57.479488 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:57.479503 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:57.480116 (+ 613us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:57.480194 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:57.480219 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:57.480340 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:57.480668 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 18:45:57.480890 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":396,"threads_started":1}
I20260811 18:45:57.486155 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:45:57.488413 4739 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ac1dca992594315a09eeba53f9b401b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac1dca992594315a09eeba53f9b401b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39127 } }
I20260811 18:45:57.488970 4739 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ac1dca992594315a09eeba53f9b401b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:57.489100 4739 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ac1dca992594315a09eeba53f9b401b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ac1dca992594315a09eeba53f9b401b, State: Initialized, Role: FOLLOWER
I20260811 18:45:57.489830 4739 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ac1dca992594315a09eeba53f9b401b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac1dca992594315a09eeba53f9b401b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39127 } }
I20260811 18:45:57.490090 4739 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ac1dca992594315a09eeba53f9b401b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:45:57.490243 4739 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ac1dca992594315a09eeba53f9b401b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:45:57.490451 4739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ac1dca992594315a09eeba53f9b401b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:57.493036 4739 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ac1dca992594315a09eeba53f9b401b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac1dca992594315a09eeba53f9b401b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39127 } }
I20260811 18:45:57.493578 4739 leader_election.cc:302] T 00000000000000000000000000000000 P 4ac1dca992594315a09eeba53f9b401b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ac1dca992594315a09eeba53f9b401b; no voters:
I20260811 18:45:57.494213 4739 leader_election.cc:288] T 00000000000000000000000000000000 P 4ac1dca992594315a09eeba53f9b401b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:45:57.494365 4759 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ac1dca992594315a09eeba53f9b401b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:45:57.496604 4739 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ac1dca992594315a09eeba53f9b401b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:45:57.498759 4745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:57.490836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47401 (local address 127.0.0.1:39127)
0811 18:45:57.491000 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:57.491021 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:57.491462 (+ 441us) server_negotiation.cc:407] Connection header received
0811 18:45:57.491683 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:57.491688 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:57.491765 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:57.491882 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:57.492464 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:57.493331 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:57.495286 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:45:57.495964 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:45:57.496103 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:45:57.497678 (+ 1575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:57.497691 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:57.497698 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:57.497720 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:57.497784 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:57.497805 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:57.497947 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:57.498208 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 18:45:57.498279 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 18:45:57.499709 4759 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ac1dca992594315a09eeba53f9b401b [term 1 LEADER]: Becoming Leader. State: Replica: 4ac1dca992594315a09eeba53f9b401b, State: Running, Role: LEADER
I20260811 18:45:57.500434 4759 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ac1dca992594315a09eeba53f9b401b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac1dca992594315a09eeba53f9b401b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39127 } }
I20260811 18:45:57.503621 4760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ac1dca992594315a09eeba53f9b401b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ac1dca992594315a09eeba53f9b401b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac1dca992594315a09eeba53f9b401b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39127 } } }
I20260811 18:45:57.504127 4760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ac1dca992594315a09eeba53f9b401b [sys.catalog]: This master's current role is: LEADER
I20260811 18:45:57.503608 4762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ac1dca992594315a09eeba53f9b401b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ac1dca992594315a09eeba53f9b401b. Latest consensus state: current_term: 1 leader_uuid: "4ac1dca992594315a09eeba53f9b401b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac1dca992594315a09eeba53f9b401b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39127 } } }
I20260811 18:45:57.504443 4762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ac1dca992594315a09eeba53f9b401b [sys.catalog]: This master's current role is: LEADER
I20260811 18:45:57.506695 4779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:45:57.511624 4779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:45:57.520094 4779 catalog_manager.cc:1347] Generated new cluster ID: c39f487b3ad94e97a38243a1b5ff6b60
I20260811 18:45:57.520221 4779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:45:57.532442 4779 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:45:57.534399 4779 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:45:57.551159 4779 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ac1dca992594315a09eeba53f9b401b: Generated new TSK 0
I20260811 18:45:57.552328 4779 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:45:57.562279 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 4606
W20260811 18:45:57.563539 4729 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46061
W20260811 18:45:57.564343 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 4606: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:45:57.564553 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 4606
W20260811 18:45:57.598912 4576 connection.cc:535] client connection to 127.0.0.1:39127 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39127): Connection reset by peer (error 104)
2026-08-11T18:45:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2368 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:45:57 dist-test-slave-z8x9 krb5kdc[4797](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:45:57 dist-test-slave-z8x9 krb5kdc[4797](info): set up 2 sockets
Aug 11 18:45:57 dist-test-slave-z8x9 krb5kdc[4797](info): commencing operation
krb5kdc: starting...
W20260811 18:45:59.010713 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.392s user 0.001s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:45:59 dist-test-slave-z8x9 krb5kdc[4797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:45:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:45:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:45:59.380312 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33485
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33485
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:45:59.736368 4827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:45:59.736739 4827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:45:59.736918 4827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:45:59.744395 4827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:45:59.744580 4827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:45:59.744684 4827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:45:59.744812 4827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:45:59.744892 4827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:45:59.755462 4827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33485
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:45:59.756884 4827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:45:59.758329 4827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:45:59.770540 4843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:59.770646 4838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:45:59.770679 4837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:45:59.771697 4827 server_base.cc:1034] running on GCE node
I20260811 18:45:59.772576 4827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:45:59.774101 4827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:45:59.775354 4827 hybrid_clock.cc:648] HybridClock initialized: now 1786473959775318 us; error 59 us; skew 500 ppm
I20260811 18:45:59.775719 4827 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:45:59 dist-test-slave-z8x9 krb5kdc[4797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473959, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:45:59.779150 4827 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:45:59.780740 4827 webserver.cc:460] Webserver started at http://127.0.0.1:45301/ using document root <none> and password file <none>
I20260811 18:45:59.781530 4827 fs_manager.cc:359] Metadata directory not provided
I20260811 18:45:59.781685 4827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:45:59.782016 4827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:45:59.785130 4827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "66ef8518574f4e7ba575716b4a866dad"
format_stamp: "Formatted at 2026-08-11 18:45:59 on dist-test-slave-z8x9"
server_key: "a5fa977fec89c808154244f3111130e9"
server_key_iv: "922d16857c125095be04a5d63038628a"
server_key_version: "encryptionkey@0"
I20260811 18:45:59.786027 4827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "66ef8518574f4e7ba575716b4a866dad"
format_stamp: "Formatted at 2026-08-11 18:45:59 on dist-test-slave-z8x9"
server_key: "a5fa977fec89c808154244f3111130e9"
server_key_iv: "922d16857c125095be04a5d63038628a"
server_key_version: "encryptionkey@0"
I20260811 18:45:59.791503 4827 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 18:45:59.795356 4861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:45:59.796774 4827 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:45:59.796968 4827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "66ef8518574f4e7ba575716b4a866dad"
format_stamp: "Formatted at 2026-08-11 18:45:59 on dist-test-slave-z8x9"
server_key: "a5fa977fec89c808154244f3111130e9"
server_key_iv: "922d16857c125095be04a5d63038628a"
server_key_version: "encryptionkey@0"
I20260811 18:45:59.797245 4827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:45:59.803495 4827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:45:59.807900 4827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:45:59.808228 4827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:45:59.828441 4827 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33485
I20260811 18:45:59.828495 4950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33485 every 8 connection(s)
I20260811 18:45:59.830066 4827 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:45:59.834061 4954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:45:59.836046 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 4827
I20260811 18:45:59.836247 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:45:59.836814 31462 external_mini_cluster.cc:1428] Setting key 8fd0bd55c6a3e2223f686ed93b3b1ac3
I20260811 18:45:59.847612 4962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:59.839127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44137 (local address 127.0.0.1:33485)
0811 18:45:59.840291 (+ 1164us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:59.840309 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:59.840735 (+ 426us) server_negotiation.cc:407] Connection header received
0811 18:45:59.843291 (+ 2556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:59.843321 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:59.843790 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:59.844264 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:59.845335 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:59.845343 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:59.845359 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:59.845384 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:59.845458 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:59.845491 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:59.845732 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:59.846023 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 18:45:59.846259 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":866,"thread_start_us":528,"threads_started":1}
I20260811 18:45:59.848753 4954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66ef8518574f4e7ba575716b4a866dad: Bootstrap starting.
I20260811 18:45:59.851501 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:45:59.852280 4954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66ef8518574f4e7ba575716b4a866dad: Neither blocks nor log segments found. Creating new log.
I20260811 18:45:59.853617 4954 log.cc:824] T 00000000000000000000000000000000 P 66ef8518574f4e7ba575716b4a866dad: Log is configured to *not* fsync() on all Append() calls
I20260811 18:45:59.857208 4954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66ef8518574f4e7ba575716b4a866dad: No bootstrap required, opened a new log
I20260811 18:45:59.859504 4962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:45:59.856948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58733 (local address 127.0.0.1:33485)
0811 18:45:59.857160 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 18:45:59.857196 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0811 18:45:59.857574 (+ 378us) server_negotiation.cc:407] Connection header received
0811 18:45:59.857727 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:45:59.857735 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:45:59.857816 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:45:59.857945 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:45:59.858837 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:45:59.858843 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:45:59.858847 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:45:59.858866 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:45:59.858923 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:45:59.858943 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:45:59.859079 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 18:45:59.859122 (+ 43us) server_negotiation.cc:299] Negotiation successful
0811 18:45:59.859183 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:45:59.866001 4954 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66ef8518574f4e7ba575716b4a866dad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ef8518574f4e7ba575716b4a866dad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33485 } }
I20260811 18:45:59.866600 4954 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66ef8518574f4e7ba575716b4a866dad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:45:59.866703 4954 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66ef8518574f4e7ba575716b4a866dad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ef8518574f4e7ba575716b4a866dad, State: Initialized, Role: FOLLOWER
I20260811 18:45:59.867231 4954 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66ef8518574f4e7ba575716b4a866dad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ef8518574f4e7ba575716b4a866dad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33485 } }
I20260811 18:45:59.867451 4954 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66ef8518574f4e7ba575716b4a866dad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:45:59.867558 4954 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66ef8518574f4e7ba575716b4a866dad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:45:59.867676 4954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66ef8518574f4e7ba575716b4a866dad [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:45:59.869951 4954 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66ef8518574f4e7ba575716b4a866dad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ef8518574f4e7ba575716b4a866dad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33485 } }
I20260811 18:45:59.870427 4954 leader_election.cc:302] T 00000000000000000000000000000000 P 66ef8518574f4e7ba575716b4a866dad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66ef8518574f4e7ba575716b4a866dad; no voters:
I20260811 18:45:59.871070 4954 leader_election.cc:288] T 00000000000000000000000000000000 P 66ef8518574f4e7ba575716b4a866dad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:45:59.871232 4973 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66ef8518574f4e7ba575716b4a866dad [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:45:59.872941 4973 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66ef8518574f4e7ba575716b4a866dad [term 1 LEADER]: Becoming Leader. State: Replica: 66ef8518574f4e7ba575716b4a866dad, State: Running, Role: LEADER
I20260811 18:45:59.873562 4954 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66ef8518574f4e7ba575716b4a866dad [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:45:59.873582 4973 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66ef8518574f4e7ba575716b4a866dad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ef8518574f4e7ba575716b4a866dad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33485 } }
I20260811 18:45:59.877740 4976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66ef8518574f4e7ba575716b4a866dad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66ef8518574f4e7ba575716b4a866dad. Latest consensus state: current_term: 1 leader_uuid: "66ef8518574f4e7ba575716b4a866dad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ef8518574f4e7ba575716b4a866dad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33485 } } }
I20260811 18:45:59.878249 4976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66ef8518574f4e7ba575716b4a866dad [sys.catalog]: This master's current role is: LEADER
I20260811 18:45:59.878854 4975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66ef8518574f4e7ba575716b4a866dad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66ef8518574f4e7ba575716b4a866dad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ef8518574f4e7ba575716b4a866dad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33485 } } }
I20260811 18:45:59.879216 4975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66ef8518574f4e7ba575716b4a866dad [sys.catalog]: This master's current role is: LEADER
I20260811 18:45:59.879164 4986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:45:59.889868 4986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:45:59.899833 4986 catalog_manager.cc:1347] Generated new cluster ID: 4472712cc870462d95b74c6bce01b5bb
I20260811 18:45:59.899976 4986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:45:59.911319 4986 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:45:59.911850 4986 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:45:59.922562 4986 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66ef8518574f4e7ba575716b4a866dad: Generated new TSK 0
I20260811 18:45:59.923609 4986 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:45:59.931015 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 4827
W20260811 18:45:59.932130 4945 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44137
W20260811 18:45:59.932825 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 4827: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:45:59.932993 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 4827
2026-08-11T18:45:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2363 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:46:00 dist-test-slave-z8x9 krb5kdc[5012](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:46:00 dist-test-slave-z8x9 krb5kdc[5012](info): set up 2 sockets
Aug 11 18:46:00 dist-test-slave-z8x9 krb5kdc[5012](info): commencing operation
krb5kdc: starting...
W20260811 18:46:01.360318 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.378s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:46:01 dist-test-slave-z8x9 krb5kdc[5012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:46:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:46:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:46:01.729127 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36415
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:46:02.082237 5040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:46:02.082538 5040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:46:02.082741 5040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:46:02.089985 5040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:46:02.090106 5040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:46:02.090199 5040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:46:02.090318 5040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:46:02.090385 5040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:46:02.100497 5040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:46:02.101975 5040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:46:02.103408 5040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:46:02.115993 5053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:02.116125 5052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:02.116161 5055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:46:02.117066 5040 server_base.cc:1034] running on GCE node
I20260811 18:46:02.117952 5040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:46:02.119405 5040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:46:02.120651 5040 hybrid_clock.cc:648] HybridClock initialized: now 1786473962120601 us; error 57 us; skew 500 ppm
I20260811 18:46:02.121054 5040 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:46:02 dist-test-slave-z8x9 krb5kdc[5012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473962, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:46:02.124557 5040 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:46:02.126123 5040 webserver.cc:460] Webserver started at http://127.0.0.1:33955/ using document root <none> and password file <none>
I20260811 18:46:02.126832 5040 fs_manager.cc:359] Metadata directory not provided
I20260811 18:46:02.126986 5040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:46:02.127303 5040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:46:02.130400 5040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "c7927a5addf042d68b9fa79ef1b02c23"
format_stamp: "Formatted at 2026-08-11 18:46:02 on dist-test-slave-z8x9"
server_key: "6632acd55c603c4de7bee99d4c2b3aa2"
server_key_iv: "77fa20ede1bfc978a5b0091d2fade23c"
server_key_version: "encryptionkey@0"
I20260811 18:46:02.131193 5040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "c7927a5addf042d68b9fa79ef1b02c23"
format_stamp: "Formatted at 2026-08-11 18:46:02 on dist-test-slave-z8x9"
server_key: "6632acd55c603c4de7bee99d4c2b3aa2"
server_key_iv: "77fa20ede1bfc978a5b0091d2fade23c"
server_key_version: "encryptionkey@0"
I20260811 18:46:02.136365 5040 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 18:46:02.140060 5065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:46:02.141471 5040 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:46:02.141654 5040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "c7927a5addf042d68b9fa79ef1b02c23"
format_stamp: "Formatted at 2026-08-11 18:46:02 on dist-test-slave-z8x9"
server_key: "6632acd55c603c4de7bee99d4c2b3aa2"
server_key_iv: "77fa20ede1bfc978a5b0091d2fade23c"
server_key_version: "encryptionkey@0"
I20260811 18:46:02.141884 5040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:46:02.158228 5040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:46:02.162818 5040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:46:02.163137 5040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:46:02.182399 5040 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36415
I20260811 18:46:02.182443 5154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36415 every 8 connection(s)
I20260811 18:46:02.183919 5040 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:46:02.187923 5156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:46:02.194308 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 5040
I20260811 18:46:02.194520 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:46:02.194972 31462 external_mini_cluster.cc:1428] Setting key 4c1886ff764a1667cd94c3b766011088
I20260811 18:46:02.198518 5156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23: Bootstrap starting.
I20260811 18:46:02.202303 5156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23: Neither blocks nor log segments found. Creating new log.
I20260811 18:46:02.203356 5156 log.cc:824] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23: Log is configured to *not* fsync() on all Append() calls
I20260811 18:46:02.206220 5156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23: No bootstrap required, opened a new log
Aug 11 18:46:02 dist-test-slave-z8x9 krb5kdc[5012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 18:46:02.216094 5156 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7927a5addf042d68b9fa79ef1b02c23" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36415 } }
I20260811 18:46:02.216562 5156 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:46:02.216686 5156 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7927a5addf042d68b9fa79ef1b02c23, State: Initialized, Role: FOLLOWER
I20260811 18:46:02.217482 5156 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7927a5addf042d68b9fa79ef1b02c23" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36415 } }
I20260811 18:46:02.217840 5156 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:46:02.218003 5156 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:46:02.218184 5156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:46:02.219592 5166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:02.196734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58987 (local address 127.0.0.1:36415)
0811 18:46:02.197517 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:02.197529 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:02.197569 (+ 40us) server_negotiation.cc:407] Connection header received
0811 18:46:02.199254 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:02.199276 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:02.199640 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:02.199958 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:02.201968 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:02.203373 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:02.205040 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:02.205777 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:02.206004 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:02.210225 (+ 4221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:02.210241 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:02.210260 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:02.210329 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:02.214677 (+ 4348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:02.215382 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:02.215391 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:02.215400 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:02.215612 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:02.215987 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:02.215992 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:02.215995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:02.216742 (+ 747us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:02.217046 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:02.217576 (+ 530us) server_negotiation.cc:299] Negotiation successful
0811 18:46:02.218062 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":382,"threads_started":1}
I20260811 18:46:02.221639 5156 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7927a5addf042d68b9fa79ef1b02c23" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36415 } }
I20260811 18:46:02.222286 5156 leader_election.cc:302] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7927a5addf042d68b9fa79ef1b02c23; no voters:
I20260811 18:46:02.222990 5156 leader_election.cc:288] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:46:02.223119 5169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:46:02.223863 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:46:02.225121 5156 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:46:02.230871 5169 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23 [term 1 LEADER]: Becoming Leader. State: Replica: c7927a5addf042d68b9fa79ef1b02c23, State: Running, Role: LEADER
I20260811 18:46:02.231535 5169 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7927a5addf042d68b9fa79ef1b02c23" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36415 } }
I20260811 18:46:02.238864 5172 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7927a5addf042d68b9fa79ef1b02c23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7927a5addf042d68b9fa79ef1b02c23" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36415 } } }
I20260811 18:46:02.239300 5172 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:02.240289 5173 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7927a5addf042d68b9fa79ef1b02c23. Latest consensus state: current_term: 1 leader_uuid: "c7927a5addf042d68b9fa79ef1b02c23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7927a5addf042d68b9fa79ef1b02c23" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36415 } } }
I20260811 18:46:02.241333 5173 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23 [sys.catalog]: This master's current role is: LEADER
W20260811 18:46:02.246292 5212 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:46:02.246450 5212 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:46:02.246675 5204 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:46:02.248909 5204 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:46:02.249485 5166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:02.231786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33855 (local address 127.0.0.1:36415)
0811 18:46:02.232004 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:02.232013 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:02.232041 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:46:02.232254 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:02.232260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:02.232338 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:02.232409 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:02.235036 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:02.236539 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:02.238641 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:02.239148 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:02.239305 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:02.242835 (+ 3530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:02.242841 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:02.242850 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:02.242882 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:02.246526 (+ 3644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:02.247287 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:02.247293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:02.247300 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:02.247467 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:02.247882 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:02.247887 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:02.247891 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:02.248219 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:02.248492 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:02.248840 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 18:46:02.249094 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 18:46:02.258301 5204 catalog_manager.cc:1347] Generated new cluster ID: 2af5188ce8b64050a52a2bb15a381084
I20260811 18:46:02.258441 5204 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:46:02.275666 5204 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:46:02.277029 5204 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:46:02.286468 5204 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7927a5addf042d68b9fa79ef1b02c23: Generated new TSK 0
I20260811 18:46:02.287635 5204 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:46:02.293836 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 5040
I20260811 18:46:02.295256 5150 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:46:02.390364 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 5040
2026-08-11T18:46:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2451 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:46:02 dist-test-slave-z8x9 krb5kdc[5226](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:46:02 dist-test-slave-z8x9 krb5kdc[5226](info): set up 2 sockets
Aug 11 18:46:02 dist-test-slave-z8x9 krb5kdc[5226](info): commencing operation
krb5kdc: starting...
W20260811 18:46:03.833527 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.400s user 0.003s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:46:04 dist-test-slave-z8x9 krb5kdc[5226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:46:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:46:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:46:04.196043 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35757
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:46:04.564636 5250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:46:04.565004 5250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:46:04.565207 5250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:46:04.572934 5250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:46:04.573096 5250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:46:04.573210 5250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:46:04.573328 5250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:46:04.573405 5250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:46:04.583966 5250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:46:04.585579 5250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:46:04.587013 5250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:46:04.599478 5262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:04.599429 5264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:04.599730 5259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:46:04.600620 5250 server_base.cc:1034] running on GCE node
I20260811 18:46:04.601519 5250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:46:04.603016 5250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:46:04.604269 5250 hybrid_clock.cc:648] HybridClock initialized: now 1786473964604225 us; error 35 us; skew 500 ppm
I20260811 18:46:04.604596 5250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:46:04 dist-test-slave-z8x9 krb5kdc[5226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473964, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:46:04.608096 5250 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:46:04.609615 5250 webserver.cc:460] Webserver started at http://127.0.0.1:40401/ using document root <none> and password file <none>
I20260811 18:46:04.610308 5250 fs_manager.cc:359] Metadata directory not provided
I20260811 18:46:04.610446 5250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:46:04.610760 5250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:46:04.613960 5250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "77f07aa27e344330b64ca8c531a110cb"
format_stamp: "Formatted at 2026-08-11 18:46:04 on dist-test-slave-z8x9"
server_key: "78f691eb2c563da95b0c386fc6ebf405"
server_key_iv: "d8696b1da4d543990768246ca8e9da28"
server_key_version: "encryptionkey@0"
I20260811 18:46:04.614749 5250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "77f07aa27e344330b64ca8c531a110cb"
format_stamp: "Formatted at 2026-08-11 18:46:04 on dist-test-slave-z8x9"
server_key: "78f691eb2c563da95b0c386fc6ebf405"
server_key_iv: "d8696b1da4d543990768246ca8e9da28"
server_key_version: "encryptionkey@0"
I20260811 18:46:04.620124 5250 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 18:46:04.623623 5280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:46:04.624931 5250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 18:46:04.625116 5250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "77f07aa27e344330b64ca8c531a110cb"
format_stamp: "Formatted at 2026-08-11 18:46:04 on dist-test-slave-z8x9"
server_key: "78f691eb2c563da95b0c386fc6ebf405"
server_key_iv: "d8696b1da4d543990768246ca8e9da28"
server_key_version: "encryptionkey@0"
I20260811 18:46:04.625371 5250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:46:04.648723 5250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:46:04.653465 5250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:46:04.653784 5250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:46:04.676563 5250 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35757
I20260811 18:46:04.676587 5372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35757 every 8 connection(s)
I20260811 18:46:04.678103 5250 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:46:04.681906 5373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:46:04.684594 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 5250
I20260811 18:46:04.684785 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:46:04.685371 31462 external_mini_cluster.cc:1428] Setting key 52dcbbc1067c178371261245ecc1de2f
I20260811 18:46:04.694293 5373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77f07aa27e344330b64ca8c531a110cb: Bootstrap starting.
I20260811 18:46:04.698208 5373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77f07aa27e344330b64ca8c531a110cb: Neither blocks nor log segments found. Creating new log.
I20260811 18:46:04.699146 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:04.687294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52657 (local address 127.0.0.1:35757)
0811 18:46:04.688136 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:04.688147 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:04.688188 (+ 41us) server_negotiation.cc:407] Connection header received
0811 18:46:04.689880 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:04.689902 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:04.690266 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:04.690597 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:04.691221 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:04.692584 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:04.694532 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:04.695215 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:04.695358 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:04.696672 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:04.696680 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:04.696694 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:46:04.697290 (+ 596us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:04.697367 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:46:04.697391 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:04.697479 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:04.697803 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 18:46:04.698110 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":422,"threads_started":1}
I20260811 18:46:04.699383 5373 log.cc:824] T 00000000000000000000000000000000 P 77f07aa27e344330b64ca8c531a110cb: Log is configured to *not* fsync() on all Append() calls
I20260811 18:46:04.702802 5373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77f07aa27e344330b64ca8c531a110cb: No bootstrap required, opened a new log
I20260811 18:46:04.703219 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:46:04.711501 5373 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77f07aa27e344330b64ca8c531a110cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77f07aa27e344330b64ca8c531a110cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35757 } }
I20260811 18:46:04.711905 5373 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77f07aa27e344330b64ca8c531a110cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:46:04.712049 5373 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77f07aa27e344330b64ca8c531a110cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77f07aa27e344330b64ca8c531a110cb, State: Initialized, Role: FOLLOWER
I20260811 18:46:04.712765 5373 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77f07aa27e344330b64ca8c531a110cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77f07aa27e344330b64ca8c531a110cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35757 } }
I20260811 18:46:04.713093 5373 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77f07aa27e344330b64ca8c531a110cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:46:04.713341 5373 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77f07aa27e344330b64ca8c531a110cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:46:04.713527 5373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77f07aa27e344330b64ca8c531a110cb [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:46:04.715142 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:04.707759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42999 (local address 127.0.0.1:35757)
0811 18:46:04.707959 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:04.707966 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:04.708350 (+ 384us) server_negotiation.cc:407] Connection header received
0811 18:46:04.708523 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:04.708534 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:04.708623 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:04.708742 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:04.709673 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:04.710814 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:04.712303 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:04.712980 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:04.713124 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:04.714355 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:04.714361 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:04.714366 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:46:04.714389 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:04.714447 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:46:04.714473 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:04.714579 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:04.714695 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 18:46:04.714796 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:46:04.716710 5373 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77f07aa27e344330b64ca8c531a110cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77f07aa27e344330b64ca8c531a110cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35757 } }
I20260811 18:46:04.717240 5373 leader_election.cc:302] T 00000000000000000000000000000000 P 77f07aa27e344330b64ca8c531a110cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77f07aa27e344330b64ca8c531a110cb; no voters:
I20260811 18:46:04.717790 5373 leader_election.cc:288] T 00000000000000000000000000000000 P 77f07aa27e344330b64ca8c531a110cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:46:04.717927 5388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77f07aa27e344330b64ca8c531a110cb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:46:04.718650 5388 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77f07aa27e344330b64ca8c531a110cb [term 1 LEADER]: Becoming Leader. State: Replica: 77f07aa27e344330b64ca8c531a110cb, State: Running, Role: LEADER
I20260811 18:46:04.719234 5388 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77f07aa27e344330b64ca8c531a110cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77f07aa27e344330b64ca8c531a110cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35757 } }
I20260811 18:46:04.720009 5373 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77f07aa27e344330b64ca8c531a110cb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:46:04.722289 5389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77f07aa27e344330b64ca8c531a110cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77f07aa27e344330b64ca8c531a110cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77f07aa27e344330b64ca8c531a110cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35757 } } }
I20260811 18:46:04.722783 5389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77f07aa27e344330b64ca8c531a110cb [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:04.722832 5391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77f07aa27e344330b64ca8c531a110cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77f07aa27e344330b64ca8c531a110cb. Latest consensus state: current_term: 1 leader_uuid: "77f07aa27e344330b64ca8c531a110cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77f07aa27e344330b64ca8c531a110cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35757 } } }
I20260811 18:46:04.723158 5391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77f07aa27e344330b64ca8c531a110cb [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:04.724759 5399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:46:04.730700 5399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:46:04.740473 5399 catalog_manager.cc:1347] Generated new cluster ID: 2004316e3bf24f359b412c181d96a571
I20260811 18:46:04.740622 5399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:46:04.774761 5399 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:46:04.776904 5399 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:46:04.795917 5399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77f07aa27e344330b64ca8c531a110cb: Generated new TSK 0
I20260811 18:46:04.797058 5399 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:46:04.811446 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 5250
W20260811 18:46:04.812728 5365 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:52657
W20260811 18:46:04.813551 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 5250: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:46:04.813715 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 5250
2026-08-11T18:46:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2428 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:46:04 dist-test-slave-z8x9 krb5kdc[5425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:46:04 dist-test-slave-z8x9 krb5kdc[5425](info): set up 2 sockets
Aug 11 18:46:04 dist-test-slave-z8x9 krb5kdc[5425](info): commencing operation
krb5kdc: starting...
W20260811 18:46:06.244482 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.382s user 0.000s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:46:06 dist-test-slave-z8x9 krb5kdc[5425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:46:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:46:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:46:06.620213 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40775
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:46:06.981936 5447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:46:06.982308 5447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:46:06.982477 5447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:46:06.989836 5447 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:46:06.989993 5447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:46:06.990101 5447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:46:06.990228 5447 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:46:06.990309 5447 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:46:07.000483 5447 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:46:07.001896 5447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:46:07.003247 5447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:46:07.015643 5467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:07.015801 5461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:07.015738 5463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:46:07.016686 5447 server_base.cc:1034] running on GCE node
I20260811 18:46:07.017714 5447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:46:07.019233 5447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:46:07.020493 5447 hybrid_clock.cc:648] HybridClock initialized: now 1786473967020457 us; error 30 us; skew 500 ppm
I20260811 18:46:07.020864 5447 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:46:07 dist-test-slave-z8x9 krb5kdc[5425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473967, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:46:07.024319 5447 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:46:07.025979 5447 webserver.cc:460] Webserver started at http://127.0.0.1:39933/ using document root <none> and password file <none>
I20260811 18:46:07.026719 5447 fs_manager.cc:359] Metadata directory not provided
I20260811 18:46:07.026885 5447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:46:07.027204 5447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:46:07.030388 5447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "34e54cee0b1747f587290988fcba8039"
format_stamp: "Formatted at 2026-08-11 18:46:07 on dist-test-slave-z8x9"
server_key: "ca1263318eccbd0da55e5505d271a017"
server_key_iv: "4d16659a361baf7066100cc102f1c76e"
server_key_version: "encryptionkey@0"
I20260811 18:46:07.031217 5447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "34e54cee0b1747f587290988fcba8039"
format_stamp: "Formatted at 2026-08-11 18:46:07 on dist-test-slave-z8x9"
server_key: "ca1263318eccbd0da55e5505d271a017"
server_key_iv: "4d16659a361baf7066100cc102f1c76e"
server_key_version: "encryptionkey@0"
I20260811 18:46:07.036677 5447 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 18:46:07.040256 5481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:46:07.041672 5447 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:46:07.041880 5447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "34e54cee0b1747f587290988fcba8039"
format_stamp: "Formatted at 2026-08-11 18:46:07 on dist-test-slave-z8x9"
server_key: "ca1263318eccbd0da55e5505d271a017"
server_key_iv: "4d16659a361baf7066100cc102f1c76e"
server_key_version: "encryptionkey@0"
I20260811 18:46:07.042094 5447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:46:07.047948 5447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:46:07.052554 5447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:46:07.052888 5447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:46:07.072113 5447 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40775
I20260811 18:46:07.072158 5582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40775 every 8 connection(s)
I20260811 18:46:07.073658 5447 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:46:07.075747 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 5447
I20260811 18:46:07.075948 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:46:07.076648 31462 external_mini_cluster.cc:1428] Setting key e038491ba4e697278f747f2ff85b8a3d
I20260811 18:46:07.077998 5584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:46:07.085594 5590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:07.078509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45891 (local address 127.0.0.1:40775)
0811 18:46:07.079325 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:07.079343 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:07.079396 (+ 53us) server_negotiation.cc:407] Connection header received
0811 18:46:07.081476 (+ 2080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:07.081504 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:07.081960 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:07.082401 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:07.083394 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:07.083403 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:07.083427 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:46:07.083444 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:07.083541 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:46:07.083570 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:07.083788 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:07.083930 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 18:46:07.084124 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":336,"threads_started":1}
I20260811 18:46:07.089540 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:46:07.093423 5584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34e54cee0b1747f587290988fcba8039: Bootstrap starting.
I20260811 18:46:07.097404 5584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34e54cee0b1747f587290988fcba8039: Neither blocks nor log segments found. Creating new log.
I20260811 18:46:07.097916 5590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:07.095203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33641 (local address 127.0.0.1:40775)
0811 18:46:07.095435 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:07.095442 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:07.095832 (+ 390us) server_negotiation.cc:407] Connection header received
0811 18:46:07.095975 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:07.095981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:07.096044 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:07.096155 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:07.097096 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:07.097106 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:07.097113 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:46:07.097149 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:07.097239 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:46:07.097272 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:07.097406 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:07.097516 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 18:46:07.097579 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 18:46:07.098721 5584 log.cc:824] T 00000000000000000000000000000000 P 34e54cee0b1747f587290988fcba8039: Log is configured to *not* fsync() on all Append() calls
I20260811 18:46:07.101601 5584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34e54cee0b1747f587290988fcba8039: No bootstrap required, opened a new log
I20260811 18:46:07.108824 5584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34e54cee0b1747f587290988fcba8039 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e54cee0b1747f587290988fcba8039" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40775 } }
I20260811 18:46:07.109294 5584 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34e54cee0b1747f587290988fcba8039 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:46:07.109405 5584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34e54cee0b1747f587290988fcba8039 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34e54cee0b1747f587290988fcba8039, State: Initialized, Role: FOLLOWER
I20260811 18:46:07.110015 5584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34e54cee0b1747f587290988fcba8039 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e54cee0b1747f587290988fcba8039" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40775 } }
I20260811 18:46:07.110272 5584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34e54cee0b1747f587290988fcba8039 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:46:07.110416 5584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34e54cee0b1747f587290988fcba8039 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:46:07.110599 5584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34e54cee0b1747f587290988fcba8039 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:46:07.112823 5584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34e54cee0b1747f587290988fcba8039 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e54cee0b1747f587290988fcba8039" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40775 } }
I20260811 18:46:07.113444 5584 leader_election.cc:302] T 00000000000000000000000000000000 P 34e54cee0b1747f587290988fcba8039 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34e54cee0b1747f587290988fcba8039; no voters:
I20260811 18:46:07.114140 5584 leader_election.cc:288] T 00000000000000000000000000000000 P 34e54cee0b1747f587290988fcba8039 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:46:07.114253 5602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34e54cee0b1747f587290988fcba8039 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:46:07.114951 5602 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34e54cee0b1747f587290988fcba8039 [term 1 LEADER]: Becoming Leader. State: Replica: 34e54cee0b1747f587290988fcba8039, State: Running, Role: LEADER
I20260811 18:46:07.115558 5602 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34e54cee0b1747f587290988fcba8039 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e54cee0b1747f587290988fcba8039" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40775 } }
I20260811 18:46:07.116655 5584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34e54cee0b1747f587290988fcba8039 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:46:07.118777 5604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34e54cee0b1747f587290988fcba8039 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34e54cee0b1747f587290988fcba8039" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e54cee0b1747f587290988fcba8039" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40775 } } }
I20260811 18:46:07.118917 5603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34e54cee0b1747f587290988fcba8039 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34e54cee0b1747f587290988fcba8039. Latest consensus state: current_term: 1 leader_uuid: "34e54cee0b1747f587290988fcba8039" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34e54cee0b1747f587290988fcba8039" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40775 } } }
I20260811 18:46:07.119273 5604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34e54cee0b1747f587290988fcba8039 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:07.119339 5603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34e54cee0b1747f587290988fcba8039 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:07.120860 5608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:46:07.127810 5608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:46:07.138846 5608 catalog_manager.cc:1347] Generated new cluster ID: 074b7f8809c144da8817f46002491ae6
I20260811 18:46:07.138990 5608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:46:07.152487 5608 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:46:07.152961 5608 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:46:07.164358 5608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34e54cee0b1747f587290988fcba8039: Generated new TSK 0
I20260811 18:46:07.165315 5608 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:46:07.176301 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 5447
W20260811 18:46:07.177594 5577 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45891
W20260811 18:46:07.178360 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 5447: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:46:07.178514 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 5447
2026-08-11T18:46:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2367 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:46:07 dist-test-slave-z8x9 krb5kdc[5650](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:46:07 dist-test-slave-z8x9 krb5kdc[5650](info): set up 2 sockets
Aug 11 18:46:07 dist-test-slave-z8x9 krb5kdc[5650](info): commencing operation
krb5kdc: starting...
W20260811 18:46:08.613785 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.383s user 0.003s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:46:08 dist-test-slave-z8x9 krb5kdc[5650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:46:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:46:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:46:08.976615 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38777
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:46:09.333083 5671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:46:09.333503 5671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:46:09.333681 5671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:46:09.341223 5671 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:46:09.341379 5671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:46:09.341470 5671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:46:09.341581 5671 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:46:09.341650 5671 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:46:09.351923 5671 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:46:09.353387 5671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:46:09.354794 5671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:46:09.366786 5679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:09.366902 5680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:09.366905 5683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:46:09.368139 5671 server_base.cc:1034] running on GCE node
I20260811 18:46:09.368952 5671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:46:09.370532 5671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:46:09.371745 5671 hybrid_clock.cc:648] HybridClock initialized: now 1786473969371690 us; error 49 us; skew 500 ppm
I20260811 18:46:09.372107 5671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:46:09 dist-test-slave-z8x9 krb5kdc[5650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473969, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:46:09.386472 5671 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:46:09.388283 5671 webserver.cc:460] Webserver started at http://127.0.0.1:43241/ using document root <none> and password file <none>
I20260811 18:46:09.389067 5671 fs_manager.cc:359] Metadata directory not provided
I20260811 18:46:09.389267 5671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:46:09.389631 5671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:46:09.393239 5671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "fef94363e16b4266a0cc2bc1783c0337"
format_stamp: "Formatted at 2026-08-11 18:46:09 on dist-test-slave-z8x9"
server_key: "e527957d950efaeeeb9debf014d365a3"
server_key_iv: "bd19cac407bbb365978a95d98f73439d"
server_key_version: "encryptionkey@0"
I20260811 18:46:09.394270 5671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "fef94363e16b4266a0cc2bc1783c0337"
format_stamp: "Formatted at 2026-08-11 18:46:09 on dist-test-slave-z8x9"
server_key: "e527957d950efaeeeb9debf014d365a3"
server_key_iv: "bd19cac407bbb365978a95d98f73439d"
server_key_version: "encryptionkey@0"
I20260811 18:46:09.400740 5671 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260811 18:46:09.404538 5695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:46:09.406028 5671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:46:09.406227 5671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "fef94363e16b4266a0cc2bc1783c0337"
format_stamp: "Formatted at 2026-08-11 18:46:09 on dist-test-slave-z8x9"
server_key: "e527957d950efaeeeb9debf014d365a3"
server_key_iv: "bd19cac407bbb365978a95d98f73439d"
server_key_version: "encryptionkey@0"
I20260811 18:46:09.406443 5671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:46:09.434092 5671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:46:09.438352 5671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:46:09.438660 5671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:46:09.457826 5671 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38777
I20260811 18:46:09.457847 5797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38777 every 8 connection(s)
I20260811 18:46:09.459285 5671 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:46:09.462782 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 5671
I20260811 18:46:09.462980 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:46:09.463263 5799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:46:09.463518 31462 external_mini_cluster.cc:1428] Setting key cf0dbf57bf24d0c4c1b7c1da3ef94f89
I20260811 18:46:09.478178 5799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fef94363e16b4266a0cc2bc1783c0337: Bootstrap starting.
Aug 11 18:46:09 dist-test-slave-z8x9 krb5kdc[5650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 18:46:09.482173 5799 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fef94363e16b4266a0cc2bc1783c0337: Neither blocks nor log segments found. Creating new log.
I20260811 18:46:09.483577 5799 log.cc:824] T 00000000000000000000000000000000 P fef94363e16b4266a0cc2bc1783c0337: Log is configured to *not* fsync() on all Append() calls
I20260811 18:46:09.487311 5799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fef94363e16b4266a0cc2bc1783c0337: No bootstrap required, opened a new log
I20260811 18:46:09.489066 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:09.465504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:32917 (local address 127.0.0.1:38777)
0811 18:46:09.466340 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:09.466366 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:09.466420 (+ 54us) server_negotiation.cc:407] Connection header received
0811 18:46:09.468445 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:09.468472 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:09.468982 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:09.469433 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:09.470816 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:09.472829 (+ 2013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:09.474185 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:09.475302 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:09.475501 (+ 199us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:09.479861 (+ 4360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:09.479873 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:09.479894 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:09.479953 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:09.484033 (+ 4080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:09.485168 (+ 1135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:09.485177 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:09.485212 (+ 35us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:09.485436 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:09.485898 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:09.485903 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:09.485905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:09.486615 (+ 710us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:09.486942 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:09.487355 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 18:46:09.487703 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":311,"threads_started":1}
I20260811 18:46:09.493149 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:46:09.496464 5799 raft_consensus.cc:348] T 00000000000000000000000000000000 P fef94363e16b4266a0cc2bc1783c0337 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef94363e16b4266a0cc2bc1783c0337" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38777 } }
I20260811 18:46:09.496868 5799 raft_consensus.cc:374] T 00000000000000000000000000000000 P fef94363e16b4266a0cc2bc1783c0337 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:46:09.496974 5799 raft_consensus.cc:729] T 00000000000000000000000000000000 P fef94363e16b4266a0cc2bc1783c0337 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef94363e16b4266a0cc2bc1783c0337, State: Initialized, Role: FOLLOWER
I20260811 18:46:09.497709 5799 consensus_queue.cc:260] T 00000000000000000000000000000000 P fef94363e16b4266a0cc2bc1783c0337 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef94363e16b4266a0cc2bc1783c0337" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38777 } }
I20260811 18:46:09.498070 5799 raft_consensus.cc:388] T 00000000000000000000000000000000 P fef94363e16b4266a0cc2bc1783c0337 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:46:09.498339 5799 raft_consensus.cc:482] T 00000000000000000000000000000000 P fef94363e16b4266a0cc2bc1783c0337 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:46:09.498549 5799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fef94363e16b4266a0cc2bc1783c0337 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:46:09.501240 5799 raft_consensus.cc:504] T 00000000000000000000000000000000 P fef94363e16b4266a0cc2bc1783c0337 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef94363e16b4266a0cc2bc1783c0337" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38777 } }
I20260811 18:46:09.501828 5799 leader_election.cc:302] T 00000000000000000000000000000000 P fef94363e16b4266a0cc2bc1783c0337 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fef94363e16b4266a0cc2bc1783c0337; no voters:
I20260811 18:46:09.502487 5799 leader_election.cc:288] T 00000000000000000000000000000000 P fef94363e16b4266a0cc2bc1783c0337 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:46:09.502646 5819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fef94363e16b4266a0cc2bc1783c0337 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:46:09.504009 5819 raft_consensus.cc:686] T 00000000000000000000000000000000 P fef94363e16b4266a0cc2bc1783c0337 [term 1 LEADER]: Becoming Leader. State: Replica: fef94363e16b4266a0cc2bc1783c0337, State: Running, Role: LEADER
I20260811 18:46:09.504678 5819 consensus_queue.cc:237] T 00000000000000000000000000000000 P fef94363e16b4266a0cc2bc1783c0337 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef94363e16b4266a0cc2bc1783c0337" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38777 } }
I20260811 18:46:09.504980 5799 sys_catalog.cc:564] T 00000000000000000000000000000000 P fef94363e16b4266a0cc2bc1783c0337 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:46:09.514520 5822 sys_catalog.cc:455] T 00000000000000000000000000000000 P fef94363e16b4266a0cc2bc1783c0337 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fef94363e16b4266a0cc2bc1783c0337. Latest consensus state: current_term: 1 leader_uuid: "fef94363e16b4266a0cc2bc1783c0337" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef94363e16b4266a0cc2bc1783c0337" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38777 } } }
I20260811 18:46:09.515000 5822 sys_catalog.cc:458] T 00000000000000000000000000000000 P fef94363e16b4266a0cc2bc1783c0337 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:09.516371 5820 sys_catalog.cc:455] T 00000000000000000000000000000000 P fef94363e16b4266a0cc2bc1783c0337 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fef94363e16b4266a0cc2bc1783c0337" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fef94363e16b4266a0cc2bc1783c0337" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38777 } } }
I20260811 18:46:09.516826 5820 sys_catalog.cc:458] T 00000000000000000000000000000000 P fef94363e16b4266a0cc2bc1783c0337 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:09.518662 5832 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:46:09.519299 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:09.498121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:50781 (local address 127.0.0.1:38777)
0811 18:46:09.498296 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:09.498326 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:09.498869 (+ 543us) server_negotiation.cc:407] Connection header received
0811 18:46:09.498985 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:09.498993 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:09.499078 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:09.499195 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:09.500604 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:09.501739 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:09.503763 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:09.504469 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:09.504624 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:09.508678 (+ 4054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:09.508688 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:09.508696 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:09.508753 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:09.513986 (+ 5233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:09.515393 (+ 1407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:09.515402 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:09.515408 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:09.515632 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:09.516397 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:09.516405 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:09.516411 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:09.516975 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:09.517377 (+ 402us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:09.518403 (+ 1026us) server_negotiation.cc:299] Negotiation successful
0811 18:46:09.518615 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 18:46:09.525818 5832 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:46:09.536101 5832 catalog_manager.cc:1347] Generated new cluster ID: 843416bf3e8a496796f35448050657d7
I20260811 18:46:09.536237 5832 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:46:09.570432 5832 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:46:09.572335 5832 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:46:09.581080 5832 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fef94363e16b4266a0cc2bc1783c0337: Generated new TSK 0
I20260811 18:46:09.582226 5832 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:46:09.593657 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 5671
I20260811 18:46:09.595582 5785 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:46:09.690371 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 5671
W20260811 18:46:09.718828 5643 connection.cc:535] client connection to 127.0.0.1:38777 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38777): Connection reset by peer (error 104)
2026-08-11T18:46:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2508 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:46:09 dist-test-slave-z8x9 krb5kdc[5868](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:46:09 dist-test-slave-z8x9 krb5kdc[5868](info): set up 2 sockets
Aug 11 18:46:09 dist-test-slave-z8x9 krb5kdc[5868](info): commencing operation
krb5kdc: starting...
W20260811 18:46:11.123557 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.385s user 0.000s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:46:11 dist-test-slave-z8x9 krb5kdc[5868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:46:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:46:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:46:11.481575 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33033
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:46:11.831501 5894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:46:11.831827 5894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:46:11.832010 5894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:46:11.839370 5894 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:46:11.839494 5894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:46:11.839627 5894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:46:11.839742 5894 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:46:11.839831 5894 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:46:11.849973 5894 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:46:11.851401 5894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:46:11.852818 5894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:46:11.865433 5900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:11.865476 5902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:11.865525 5904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:46:11.866587 5894 server_base.cc:1034] running on GCE node
I20260811 18:46:11.867563 5894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:46:11.869107 5894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:46:11.870374 5894 hybrid_clock.cc:648] HybridClock initialized: now 1786473971870355 us; error 58 us; skew 500 ppm
I20260811 18:46:11.870762 5894 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:46:11 dist-test-slave-z8x9 krb5kdc[5868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473971, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:46:11.874284 5894 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:46:11.875910 5894 webserver.cc:460] Webserver started at http://127.0.0.1:33949/ using document root <none> and password file <none>
I20260811 18:46:11.876628 5894 fs_manager.cc:359] Metadata directory not provided
I20260811 18:46:11.876796 5894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:46:11.877122 5894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:46:11.880337 5894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "8ec00e5248944803b47aee1fa5c3a7c8"
format_stamp: "Formatted at 2026-08-11 18:46:11 on dist-test-slave-z8x9"
server_key: "1d5680b08f0e83c64987e86b8dec31ee"
server_key_iv: "0ecd2d5a279ee21fc426644ccba50f8e"
server_key_version: "encryptionkey@0"
I20260811 18:46:11.881146 5894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "8ec00e5248944803b47aee1fa5c3a7c8"
format_stamp: "Formatted at 2026-08-11 18:46:11 on dist-test-slave-z8x9"
server_key: "1d5680b08f0e83c64987e86b8dec31ee"
server_key_iv: "0ecd2d5a279ee21fc426644ccba50f8e"
server_key_version: "encryptionkey@0"
I20260811 18:46:11.886610 5894 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 18:46:11.890110 5918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:46:11.891444 5894 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:46:11.891641 5894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "8ec00e5248944803b47aee1fa5c3a7c8"
format_stamp: "Formatted at 2026-08-11 18:46:11 on dist-test-slave-z8x9"
server_key: "1d5680b08f0e83c64987e86b8dec31ee"
server_key_iv: "0ecd2d5a279ee21fc426644ccba50f8e"
server_key_version: "encryptionkey@0"
I20260811 18:46:11.891846 5894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:46:11.929602 5894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:46:11.934556 5894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:46:11.934883 5894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:46:11.954761 5894 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33033
I20260811 18:46:11.954803 6029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33033 every 8 connection(s)
I20260811 18:46:11.956240 5894 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:46:11.956859 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 5894
I20260811 18:46:11.957079 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:46:11.957792 31462 external_mini_cluster.cc:1428] Setting key 377caa9aa524a9ec63adc241a7c61bc4
I20260811 18:46:11.960976 6030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:46:11.971587 6035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:11.959774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:36659 (local address 127.0.0.1:33033)
0811 18:46:11.960602 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:11.960613 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:11.960656 (+ 43us) server_negotiation.cc:407] Connection header received
0811 18:46:11.962439 (+ 1783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:11.962461 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:11.962848 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:11.963237 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:11.963843 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:11.965240 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:11.966648 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:11.967356 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:11.967574 (+ 218us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:11.969021 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:11.969029 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:11.969045 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:46:11.969748 (+ 703us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:11.969836 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:46:11.969865 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:11.970061 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:11.970382 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 18:46:11.970556 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":333,"threads_started":1}
I20260811 18:46:11.973021 6030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ec00e5248944803b47aee1fa5c3a7c8: Bootstrap starting.
I20260811 18:46:11.975705 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:46:11.976562 6030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ec00e5248944803b47aee1fa5c3a7c8: Neither blocks nor log segments found. Creating new log.
I20260811 18:46:11.977684 6030 log.cc:824] T 00000000000000000000000000000000 P 8ec00e5248944803b47aee1fa5c3a7c8: Log is configured to *not* fsync() on all Append() calls
I20260811 18:46:11.980880 6030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ec00e5248944803b47aee1fa5c3a7c8: No bootstrap required, opened a new log
I20260811 18:46:11.987483 6035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:11.980330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:50257 (local address 127.0.0.1:33033)
0811 18:46:11.980515 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:11.980521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:11.981157 (+ 636us) server_negotiation.cc:407] Connection header received
0811 18:46:11.981377 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:11.981383 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:11.981486 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:11.981604 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:11.982132 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:11.983353 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:11.984680 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:11.985433 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:11.985556 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:11.986658 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:11.986666 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:11.986671 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:46:11.986694 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:11.986740 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:46:11.986760 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:11.986885 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:11.987047 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 18:46:11.987104 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 18:46:11.990603 6030 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ec00e5248944803b47aee1fa5c3a7c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec00e5248944803b47aee1fa5c3a7c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33033 } }
I20260811 18:46:11.990967 6030 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ec00e5248944803b47aee1fa5c3a7c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:46:11.991045 6030 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ec00e5248944803b47aee1fa5c3a7c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ec00e5248944803b47aee1fa5c3a7c8, State: Initialized, Role: FOLLOWER
I20260811 18:46:11.991600 6030 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ec00e5248944803b47aee1fa5c3a7c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec00e5248944803b47aee1fa5c3a7c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33033 } }
I20260811 18:46:11.991812 6030 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ec00e5248944803b47aee1fa5c3a7c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:46:11.991914 6030 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ec00e5248944803b47aee1fa5c3a7c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:46:11.992049 6030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ec00e5248944803b47aee1fa5c3a7c8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:46:11.994581 6030 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ec00e5248944803b47aee1fa5c3a7c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec00e5248944803b47aee1fa5c3a7c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33033 } }
I20260811 18:46:11.995028 6030 leader_election.cc:302] T 00000000000000000000000000000000 P 8ec00e5248944803b47aee1fa5c3a7c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ec00e5248944803b47aee1fa5c3a7c8; no voters:
I20260811 18:46:11.995620 6030 leader_election.cc:288] T 00000000000000000000000000000000 P 8ec00e5248944803b47aee1fa5c3a7c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:46:11.995785 6052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ec00e5248944803b47aee1fa5c3a7c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:46:11.996544 6052 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ec00e5248944803b47aee1fa5c3a7c8 [term 1 LEADER]: Becoming Leader. State: Replica: 8ec00e5248944803b47aee1fa5c3a7c8, State: Running, Role: LEADER
I20260811 18:46:11.997103 6052 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ec00e5248944803b47aee1fa5c3a7c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec00e5248944803b47aee1fa5c3a7c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33033 } }
I20260811 18:46:11.997838 6030 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ec00e5248944803b47aee1fa5c3a7c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:46:12.000089 6062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ec00e5248944803b47aee1fa5c3a7c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ec00e5248944803b47aee1fa5c3a7c8. Latest consensus state: current_term: 1 leader_uuid: "8ec00e5248944803b47aee1fa5c3a7c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec00e5248944803b47aee1fa5c3a7c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33033 } } }
I20260811 18:46:12.000515 6062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ec00e5248944803b47aee1fa5c3a7c8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:12.001346 6054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ec00e5248944803b47aee1fa5c3a7c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ec00e5248944803b47aee1fa5c3a7c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec00e5248944803b47aee1fa5c3a7c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33033 } } }
I20260811 18:46:12.001778 6054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ec00e5248944803b47aee1fa5c3a7c8 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:12.008854 6068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:46:12.014647 6068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:46:12.024042 6068 catalog_manager.cc:1347] Generated new cluster ID: f031383f047f473ab1845764b5ea8a60
I20260811 18:46:12.024181 6068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:46:12.039021 6068 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:46:12.040907 6068 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:46:12.051086 6068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ec00e5248944803b47aee1fa5c3a7c8: Generated new TSK 0
I20260811 18:46:12.052184 6068 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:46:12.063133 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 5894
W20260811 18:46:12.064324 6018 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.69:36659
W20260811 18:46:12.065129 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 5894: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:46:12.065351 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 5894
W20260811 18:46:12.094415 5864 connection.cc:535] client connection to 127.0.0.1:33033 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33033): Connection reset by peer (error 104)
2026-08-11T18:46:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2375 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:46:12 dist-test-slave-z8x9 krb5kdc[6104](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:46:12 dist-test-slave-z8x9 krb5kdc[6104](info): set up 2 sockets
Aug 11 18:46:12 dist-test-slave-z8x9 krb5kdc[6104](info): commencing operation
krb5kdc: starting...
W20260811 18:46:13.499271 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.385s user 0.001s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:46:13 dist-test-slave-z8x9 krb5kdc[6104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:46:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:46:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:46:13.861233 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39575
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:46:14.215946 6121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:46:14.216322 6121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:46:14.216490 6121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:46:14.223759 6121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:46:14.223925 6121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:46:14.224040 6121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:46:14.224153 6121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:46:14.224251 6121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:46:14.234277 6121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:46:14.235627 6121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:46:14.236989 6121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:46:14.249488 6128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:14.249393 6132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:14.249449 6129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:46:14.250628 6121 server_base.cc:1034] running on GCE node
I20260811 18:46:14.251457 6121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:46:14.252856 6121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:46:14.254094 6121 hybrid_clock.cc:648] HybridClock initialized: now 1786473974254046 us; error 42 us; skew 500 ppm
I20260811 18:46:14.254472 6121 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:46:14 dist-test-slave-z8x9 krb5kdc[6104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473974, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:46:14.257860 6121 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:46:14.259443 6121 webserver.cc:460] Webserver started at http://127.0.0.1:45985/ using document root <none> and password file <none>
I20260811 18:46:14.260136 6121 fs_manager.cc:359] Metadata directory not provided
I20260811 18:46:14.260291 6121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:46:14.260613 6121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:46:14.263674 6121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "ad1989d502ef4abab3fdc1ac5e409d67"
format_stamp: "Formatted at 2026-08-11 18:46:14 on dist-test-slave-z8x9"
server_key: "01822a23c2f018b7f70607b75f0505a7"
server_key_iv: "94801770f0e6c264185ca756234f77fd"
server_key_version: "encryptionkey@0"
I20260811 18:46:14.264477 6121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "ad1989d502ef4abab3fdc1ac5e409d67"
format_stamp: "Formatted at 2026-08-11 18:46:14 on dist-test-slave-z8x9"
server_key: "01822a23c2f018b7f70607b75f0505a7"
server_key_iv: "94801770f0e6c264185ca756234f77fd"
server_key_version: "encryptionkey@0"
I20260811 18:46:14.269654 6121 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 18:46:14.273056 6145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:46:14.274407 6121 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:46:14.274597 6121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "ad1989d502ef4abab3fdc1ac5e409d67"
format_stamp: "Formatted at 2026-08-11 18:46:14 on dist-test-slave-z8x9"
server_key: "01822a23c2f018b7f70607b75f0505a7"
server_key_iv: "94801770f0e6c264185ca756234f77fd"
server_key_version: "encryptionkey@0"
I20260811 18:46:14.274804 6121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:46:14.280650 6121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:46:14.285051 6121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:46:14.285390 6121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:46:14.304728 6121 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39575
I20260811 18:46:14.304821 6239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39575 every 8 connection(s)
I20260811 18:46:14.306176 6121 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:46:14.306960 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 6121
I20260811 18:46:14.307212 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:46:14.307811 31462 external_mini_cluster.cc:1428] Setting key 2ba80009e8da329ddd2c2d9d752f2f8d
I20260811 18:46:14.310804 6240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:46:14.316622 6243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:14.309742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:51759 (local address 127.0.0.1:39575)
0811 18:46:14.310536 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:14.310548 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:14.310591 (+ 43us) server_negotiation.cc:407] Connection header received
0811 18:46:14.312368 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:14.312392 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:14.312710 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:14.313111 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:14.314499 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:14.314505 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:14.314518 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:46:14.314533 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:14.314606 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:46:14.314650 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:14.314920 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:14.315077 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 18:46:14.315293 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":347,"threads_started":1}
I20260811 18:46:14.326574 6240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad1989d502ef4abab3fdc1ac5e409d67: Bootstrap starting.
I20260811 18:46:14.327085 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:46:14.331414 6240 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad1989d502ef4abab3fdc1ac5e409d67: Neither blocks nor log segments found. Creating new log.
I20260811 18:46:14.332923 6240 log.cc:824] T 00000000000000000000000000000000 P ad1989d502ef4abab3fdc1ac5e409d67: Log is configured to *not* fsync() on all Append() calls
I20260811 18:46:14.335403 6243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:14.332796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:40545 (local address 127.0.0.1:39575)
0811 18:46:14.332986 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:14.332993 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:14.333290 (+ 297us) server_negotiation.cc:407] Connection header received
0811 18:46:14.333526 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:14.333532 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:14.333597 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:14.333721 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:14.334787 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:14.334791 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:14.334794 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:46:14.334810 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:14.334849 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:46:14.334864 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:14.334961 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:14.335127 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 18:46:14.335197 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 18:46:14.336647 6240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad1989d502ef4abab3fdc1ac5e409d67: No bootstrap required, opened a new log
I20260811 18:46:14.344281 6240 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad1989d502ef4abab3fdc1ac5e409d67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1989d502ef4abab3fdc1ac5e409d67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39575 } }
I20260811 18:46:14.344722 6240 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad1989d502ef4abab3fdc1ac5e409d67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:46:14.344877 6240 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad1989d502ef4abab3fdc1ac5e409d67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad1989d502ef4abab3fdc1ac5e409d67, State: Initialized, Role: FOLLOWER
I20260811 18:46:14.345500 6240 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad1989d502ef4abab3fdc1ac5e409d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1989d502ef4abab3fdc1ac5e409d67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39575 } }
I20260811 18:46:14.345752 6240 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad1989d502ef4abab3fdc1ac5e409d67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:46:14.345918 6240 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad1989d502ef4abab3fdc1ac5e409d67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:46:14.346103 6240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad1989d502ef4abab3fdc1ac5e409d67 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:46:14.348336 6240 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad1989d502ef4abab3fdc1ac5e409d67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1989d502ef4abab3fdc1ac5e409d67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39575 } }
I20260811 18:46:14.348846 6240 leader_election.cc:302] T 00000000000000000000000000000000 P ad1989d502ef4abab3fdc1ac5e409d67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad1989d502ef4abab3fdc1ac5e409d67; no voters:
I20260811 18:46:14.349550 6240 leader_election.cc:288] T 00000000000000000000000000000000 P ad1989d502ef4abab3fdc1ac5e409d67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:46:14.349686 6253 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad1989d502ef4abab3fdc1ac5e409d67 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:46:14.350400 6253 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad1989d502ef4abab3fdc1ac5e409d67 [term 1 LEADER]: Becoming Leader. State: Replica: ad1989d502ef4abab3fdc1ac5e409d67, State: Running, Role: LEADER
I20260811 18:46:14.350971 6253 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad1989d502ef4abab3fdc1ac5e409d67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1989d502ef4abab3fdc1ac5e409d67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39575 } }
I20260811 18:46:14.351711 6240 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad1989d502ef4abab3fdc1ac5e409d67 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:46:14.354456 6254 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1989d502ef4abab3fdc1ac5e409d67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad1989d502ef4abab3fdc1ac5e409d67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1989d502ef4abab3fdc1ac5e409d67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39575 } } }
I20260811 18:46:14.355059 6254 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1989d502ef4abab3fdc1ac5e409d67 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:14.354552 6255 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1989d502ef4abab3fdc1ac5e409d67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad1989d502ef4abab3fdc1ac5e409d67. Latest consensus state: current_term: 1 leader_uuid: "ad1989d502ef4abab3fdc1ac5e409d67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1989d502ef4abab3fdc1ac5e409d67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39575 } } }
I20260811 18:46:14.355644 6255 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1989d502ef4abab3fdc1ac5e409d67 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:14.356953 6272 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:46:14.361567 6272 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:46:14.369864 6272 catalog_manager.cc:1347] Generated new cluster ID: e406519206e3479f8070ac8a31ff025e
I20260811 18:46:14.369988 6272 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:46:14.403868 6272 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:46:14.404336 6272 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:46:14.412148 6272 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad1989d502ef4abab3fdc1ac5e409d67: Generated new TSK 0
I20260811 18:46:14.412979 6272 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:46:14.423396 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 6121
W20260811 18:46:14.424629 6233 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.69:51759
W20260811 18:46:14.425384 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 6121: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:46:14.425537 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 6121
2026-08-11T18:46:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2365 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:46:14 dist-test-slave-z8x9 krb5kdc[6288](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:46:14 dist-test-slave-z8x9 krb5kdc[6288](info): set up 2 sockets
Aug 11 18:46:14 dist-test-slave-z8x9 krb5kdc[6288](info): commencing operation
krb5kdc: starting...
W20260811 18:46:15.866719 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.387s user 0.001s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:46:16 dist-test-slave-z8x9 krb5kdc[6288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:46:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:46:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:46:16.241453 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40117
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:46:16.614187 6308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:46:16.614548 6308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:46:16.614719 6308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:46:16.621996 6308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:46:16.622164 6308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:46:16.622275 6308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:46:16.622385 6308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:46:16.622473 6308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:46:16.632455 6308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40117
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:46:16.633868 6308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:46:16.635205 6308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:46:16.647414 6314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:16.647388 6324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:16.647600 6322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:46:16.648764 6308 server_base.cc:1034] running on GCE node
I20260811 18:46:16.649734 6308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:46:16.651139 6308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:46:16.652370 6308 hybrid_clock.cc:648] HybridClock initialized: now 1786473976652322 us; error 33 us; skew 500 ppm
I20260811 18:46:16.652782 6308 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:46:16 dist-test-slave-z8x9 krb5kdc[6288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473976, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:46:16.656018 6308 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:46:16.657791 6308 webserver.cc:460] Webserver started at http://127.0.0.1:44077/ using document root <none> and password file <none>
I20260811 18:46:16.658516 6308 fs_manager.cc:359] Metadata directory not provided
I20260811 18:46:16.658670 6308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:46:16.658993 6308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:46:16.663282 6308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "1a28d3b0b3a44b9d831f32f229ad305b"
format_stamp: "Formatted at 2026-08-11 18:46:16 on dist-test-slave-z8x9"
server_key: "b29f653a6d776b16ed14868f2ef4251c"
server_key_iv: "596f21963781be6a0cd646eb9e7d8b58"
server_key_version: "encryptionkey@0"
I20260811 18:46:16.664409 6308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "1a28d3b0b3a44b9d831f32f229ad305b"
format_stamp: "Formatted at 2026-08-11 18:46:16 on dist-test-slave-z8x9"
server_key: "b29f653a6d776b16ed14868f2ef4251c"
server_key_iv: "596f21963781be6a0cd646eb9e7d8b58"
server_key_version: "encryptionkey@0"
I20260811 18:46:16.671293 6308 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.003s
I20260811 18:46:16.674851 6336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:46:16.676162 6308 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:46:16.676373 6308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "1a28d3b0b3a44b9d831f32f229ad305b"
format_stamp: "Formatted at 2026-08-11 18:46:16 on dist-test-slave-z8x9"
server_key: "b29f653a6d776b16ed14868f2ef4251c"
server_key_iv: "596f21963781be6a0cd646eb9e7d8b58"
server_key_version: "encryptionkey@0"
I20260811 18:46:16.676570 6308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:46:16.690336 6308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:46:16.694443 6308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:46:16.694747 6308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:46:16.714419 6308 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40117
I20260811 18:46:16.714460 6441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40117 every 8 connection(s)
I20260811 18:46:16.715850 6308 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:46:16.719308 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 6308
I20260811 18:46:16.719492 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:46:16.719604 6442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:46:16.720026 31462 external_mini_cluster.cc:1428] Setting key 98b54f10475d413cc73eaca504de0f36
I20260811 18:46:16.734285 6442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a28d3b0b3a44b9d831f32f229ad305b: Bootstrap starting.
Aug 11 18:46:16 dist-test-slave-z8x9 krb5kdc[6288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 18:46:16.738149 6442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a28d3b0b3a44b9d831f32f229ad305b: Neither blocks nor log segments found. Creating new log.
I20260811 18:46:16.739537 6442 log.cc:824] T 00000000000000000000000000000000 P 1a28d3b0b3a44b9d831f32f229ad305b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:46:16.743194 6442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a28d3b0b3a44b9d831f32f229ad305b: No bootstrap required, opened a new log
I20260811 18:46:16.745609 6450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:16.721917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:51819 (local address 127.0.0.1:40117)
0811 18:46:16.722786 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:16.722801 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:16.722853 (+ 52us) server_negotiation.cc:407] Connection header received
0811 18:46:16.724676 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:16.724704 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:16.725262 (+ 558us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:16.725717 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:16.727134 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:16.729109 (+ 1975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:16.730483 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:16.731601 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:16.731792 (+ 191us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:16.736078 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:16.736089 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:16.736108 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:16.736160 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:16.740267 (+ 4107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:16.741544 (+ 1277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:16.741552 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:16.741559 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:16.741765 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:16.742289 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:16.742295 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:16.742301 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:16.743004 (+ 703us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:16.743322 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:16.743903 (+ 581us) server_negotiation.cc:299] Negotiation successful
0811 18:46:16.744247 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":339,"threads_started":1}
I20260811 18:46:16.749801 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:46:16.752308 6442 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a28d3b0b3a44b9d831f32f229ad305b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a28d3b0b3a44b9d831f32f229ad305b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40117 } }
I20260811 18:46:16.752687 6442 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a28d3b0b3a44b9d831f32f229ad305b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:46:16.752813 6442 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a28d3b0b3a44b9d831f32f229ad305b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a28d3b0b3a44b9d831f32f229ad305b, State: Initialized, Role: FOLLOWER
I20260811 18:46:16.753479 6442 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a28d3b0b3a44b9d831f32f229ad305b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a28d3b0b3a44b9d831f32f229ad305b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40117 } }
I20260811 18:46:16.753737 6442 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a28d3b0b3a44b9d831f32f229ad305b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:46:16.753899 6442 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a28d3b0b3a44b9d831f32f229ad305b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:46:16.754097 6442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a28d3b0b3a44b9d831f32f229ad305b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:46:16.756628 6442 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a28d3b0b3a44b9d831f32f229ad305b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a28d3b0b3a44b9d831f32f229ad305b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40117 } }
I20260811 18:46:16.757251 6442 leader_election.cc:302] T 00000000000000000000000000000000 P 1a28d3b0b3a44b9d831f32f229ad305b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a28d3b0b3a44b9d831f32f229ad305b; no voters:
I20260811 18:46:16.757961 6442 leader_election.cc:288] T 00000000000000000000000000000000 P 1a28d3b0b3a44b9d831f32f229ad305b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:46:16.758073 6462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a28d3b0b3a44b9d831f32f229ad305b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:46:16.760644 6442 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a28d3b0b3a44b9d831f32f229ad305b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:46:16.765095 6462 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a28d3b0b3a44b9d831f32f229ad305b [term 1 LEADER]: Becoming Leader. State: Replica: 1a28d3b0b3a44b9d831f32f229ad305b, State: Running, Role: LEADER
I20260811 18:46:16.766018 6462 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a28d3b0b3a44b9d831f32f229ad305b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a28d3b0b3a44b9d831f32f229ad305b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40117 } }
I20260811 18:46:16.769632 6463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a28d3b0b3a44b9d831f32f229ad305b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a28d3b0b3a44b9d831f32f229ad305b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a28d3b0b3a44b9d831f32f229ad305b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40117 } } }
I20260811 18:46:16.769672 6465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a28d3b0b3a44b9d831f32f229ad305b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a28d3b0b3a44b9d831f32f229ad305b. Latest consensus state: current_term: 1 leader_uuid: "1a28d3b0b3a44b9d831f32f229ad305b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a28d3b0b3a44b9d831f32f229ad305b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40117 } } }
I20260811 18:46:16.770066 6463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a28d3b0b3a44b9d831f32f229ad305b [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:16.770072 6465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a28d3b0b3a44b9d831f32f229ad305b [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:16.772102 6450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:16.754370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:37189 (local address 127.0.0.1:40117)
0811 18:46:16.754538 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:16.754542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:16.755084 (+ 542us) server_negotiation.cc:407] Connection header received
0811 18:46:16.755286 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:16.755295 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:16.755374 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:16.755486 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:16.757020 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:16.757968 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:16.759720 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:16.760255 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:16.760389 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:16.763050 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:16.763055 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:16.763059 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:16.763088 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:16.766924 (+ 3836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:16.768084 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:16.768090 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:16.768094 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:16.768265 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:16.768889 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:16.768896 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:16.768901 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:16.769481 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:16.769810 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:16.771340 (+ 1530us) server_negotiation.cc:299] Negotiation successful
0811 18:46:16.771583 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:46:16.780967 6491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:46:16.786502 6491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:46:16.795296 6491 catalog_manager.cc:1347] Generated new cluster ID: a3ac50ef15704b6ba938629cc7c73c1e
I20260811 18:46:16.795425 6491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:46:16.819103 6491 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:46:16.820528 6491 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:46:16.831642 6491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a28d3b0b3a44b9d831f32f229ad305b: Generated new TSK 0
I20260811 18:46:16.832562 6491 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:46:16.848678 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 6308
I20260811 18:46:16.849977 6435 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:46:16.951206 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 6308
W20260811 18:46:16.982828 6286 connection.cc:535] client connection to 127.0.0.1:40117 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40117): Connection reset by peer (error 104)
2026-08-11T18:46:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2522 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:46:17 dist-test-slave-z8x9 krb5kdc[6509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:46:17 dist-test-slave-z8x9 krb5kdc[6509](info): set up 2 sockets
Aug 11 18:46:17 dist-test-slave-z8x9 krb5kdc[6509](info): commencing operation
krb5kdc: starting...
W20260811 18:46:18.399771 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.397s user 0.001s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:46:18 dist-test-slave-z8x9 krb5kdc[6509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:46:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:46:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:46:18.769024 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33699
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:46:19.117691 6526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:46:19.118017 6526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:46:19.118220 6526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:46:19.126190 6526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:46:19.126355 6526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:46:19.126430 6526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:46:19.126495 6526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:46:19.126541 6526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:46:19.136854 6526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33699
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:46:19.138338 6526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:46:19.139700 6526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:46:19.152408 6538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:19.152338 6539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:19.152365 6543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:46:19.153086 6526 server_base.cc:1034] running on GCE node
I20260811 18:46:19.154014 6526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:46:19.155517 6526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:46:19.156728 6526 hybrid_clock.cc:648] HybridClock initialized: now 1786473979156687 us; error 40 us; skew 500 ppm
I20260811 18:46:19.157109 6526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:46:19 dist-test-slave-z8x9 krb5kdc[6509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473979, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:46:19.160568 6526 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:46:19.162284 6526 webserver.cc:460] Webserver started at http://127.0.0.1:38051/ using document root <none> and password file <none>
I20260811 18:46:19.163041 6526 fs_manager.cc:359] Metadata directory not provided
I20260811 18:46:19.163197 6526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:46:19.163527 6526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:46:19.166724 6526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "66a45620a844484ab898b051a2c9741f"
format_stamp: "Formatted at 2026-08-11 18:46:19 on dist-test-slave-z8x9"
server_key: "c792b189eb634907e34c7041c0f69dd3"
server_key_iv: "8e394d21212d22b7c64db9684f4ea2e4"
server_key_version: "encryptionkey@0"
I20260811 18:46:19.167533 6526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "66a45620a844484ab898b051a2c9741f"
format_stamp: "Formatted at 2026-08-11 18:46:19 on dist-test-slave-z8x9"
server_key: "c792b189eb634907e34c7041c0f69dd3"
server_key_iv: "8e394d21212d22b7c64db9684f4ea2e4"
server_key_version: "encryptionkey@0"
I20260811 18:46:19.172931 6526 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 18:46:19.176424 6555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:46:19.177807 6526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:46:19.178025 6526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "66a45620a844484ab898b051a2c9741f"
format_stamp: "Formatted at 2026-08-11 18:46:19 on dist-test-slave-z8x9"
server_key: "c792b189eb634907e34c7041c0f69dd3"
server_key_iv: "8e394d21212d22b7c64db9684f4ea2e4"
server_key_version: "encryptionkey@0"
I20260811 18:46:19.178239 6526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:46:19.204618 6526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:46:19.208734 6526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:46:19.209055 6526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:46:19.228648 6526 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33699
I20260811 18:46:19.228669 6670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33699 every 8 connection(s)
I20260811 18:46:19.230154 6526 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:46:19.231916 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 6526
I20260811 18:46:19.232102 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:46:19.232720 31462 external_mini_cluster.cc:1428] Setting key edb89ba3c149632dc9665a6beadcb7f9
I20260811 18:46:19.234829 6672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:46:19.245868 6688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:19.234604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:35809 (local address 127.0.0.1:33699)
0811 18:46:19.235415 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:19.235427 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:19.235467 (+ 40us) server_negotiation.cc:407] Connection header received
0811 18:46:19.237097 (+ 1630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:19.237117 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:19.237594 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:19.238000 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:19.238537 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:19.239866 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:19.241200 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:19.241888 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:19.242093 (+ 205us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:19.243447 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:19.243455 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:19.243471 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:46:19.244132 (+ 661us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:19.244213 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:46:19.244241 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:19.244415 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:19.244696 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 18:46:19.244868 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":370,"threads_started":1}
I20260811 18:46:19.246412 6672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66a45620a844484ab898b051a2c9741f: Bootstrap starting.
I20260811 18:46:19.249943 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:46:19.250203 6672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66a45620a844484ab898b051a2c9741f: Neither blocks nor log segments found. Creating new log.
I20260811 18:46:19.251202 6672 log.cc:824] T 00000000000000000000000000000000 P 66a45620a844484ab898b051a2c9741f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:46:19.254143 6672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66a45620a844484ab898b051a2c9741f: No bootstrap required, opened a new log
I20260811 18:46:19.261685 6688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:19.254595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:38459 (local address 127.0.0.1:33699)
0811 18:46:19.254839 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:19.254858 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:19.255367 (+ 509us) server_negotiation.cc:407] Connection header received
0811 18:46:19.255513 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:19.255518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:19.255599 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:19.255731 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:19.256215 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:19.257512 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:19.258848 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:19.259598 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:19.259752 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:19.260868 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:19.260875 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:19.260880 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:46:19.260907 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:19.260953 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:46:19.260972 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:19.261099 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:19.261279 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 18:46:19.261340 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:46:19.264253 6672 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66a45620a844484ab898b051a2c9741f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66a45620a844484ab898b051a2c9741f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33699 } }
I20260811 18:46:19.264653 6672 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66a45620a844484ab898b051a2c9741f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:46:19.264782 6672 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66a45620a844484ab898b051a2c9741f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a45620a844484ab898b051a2c9741f, State: Initialized, Role: FOLLOWER
I20260811 18:46:19.265436 6672 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66a45620a844484ab898b051a2c9741f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66a45620a844484ab898b051a2c9741f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33699 } }
I20260811 18:46:19.265676 6672 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66a45620a844484ab898b051a2c9741f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:46:19.265849 6672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66a45620a844484ab898b051a2c9741f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:46:19.266021 6672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66a45620a844484ab898b051a2c9741f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:46:19.268272 6672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66a45620a844484ab898b051a2c9741f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66a45620a844484ab898b051a2c9741f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33699 } }
I20260811 18:46:19.268895 6672 leader_election.cc:302] T 00000000000000000000000000000000 P 66a45620a844484ab898b051a2c9741f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66a45620a844484ab898b051a2c9741f; no voters:
I20260811 18:46:19.269476 6672 leader_election.cc:288] T 00000000000000000000000000000000 P 66a45620a844484ab898b051a2c9741f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:46:19.269625 6700 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66a45620a844484ab898b051a2c9741f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:46:19.270349 6700 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66a45620a844484ab898b051a2c9741f [term 1 LEADER]: Becoming Leader. State: Replica: 66a45620a844484ab898b051a2c9741f, State: Running, Role: LEADER
I20260811 18:46:19.270921 6700 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66a45620a844484ab898b051a2c9741f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66a45620a844484ab898b051a2c9741f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33699 } }
I20260811 18:46:19.271870 6672 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66a45620a844484ab898b051a2c9741f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:46:19.274242 6701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a45620a844484ab898b051a2c9741f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66a45620a844484ab898b051a2c9741f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66a45620a844484ab898b051a2c9741f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33699 } } }
I20260811 18:46:19.274718 6701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a45620a844484ab898b051a2c9741f [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:19.277112 6716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:46:19.276921 6704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a45620a844484ab898b051a2c9741f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66a45620a844484ab898b051a2c9741f. Latest consensus state: current_term: 1 leader_uuid: "66a45620a844484ab898b051a2c9741f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66a45620a844484ab898b051a2c9741f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33699 } } }
I20260811 18:46:19.277289 6704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a45620a844484ab898b051a2c9741f [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:19.282497 6716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:46:19.291302 6716 catalog_manager.cc:1347] Generated new cluster ID: e4e76285ae984194844eff95edaae83d
I20260811 18:46:19.291419 6716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:46:19.307480 6716 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:46:19.309394 6716 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:46:19.317370 6716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66a45620a844484ab898b051a2c9741f: Generated new TSK 0
I20260811 18:46:19.318374 6716 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:46:19.331574 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 6526
W20260811 18:46:19.333120 6665 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.69:35809
W20260811 18:46:19.334076 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 6526: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:46:19.334241 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 6526
W20260811 18:46:19.365644 6506 connection.cc:535] client connection to 127.0.0.1:33699 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33699): Connection reset by peer (error 104)
2026-08-11T18:46:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2382 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:46:19 dist-test-slave-z8x9 krb5kdc[6745](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:46:19 dist-test-slave-z8x9 krb5kdc[6745](info): set up 2 sockets
Aug 11 18:46:19 dist-test-slave-z8x9 krb5kdc[6745](info): commencing operation
krb5kdc: starting...
W20260811 18:46:20.778402 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.393s user 0.003s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:46:21 dist-test-slave-z8x9 krb5kdc[6745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:46:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:46:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:46:21.132884 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40137
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:46:21.483356 6771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:46:21.483676 6771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:46:21.483845 6771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:46:21.491437 6771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:46:21.491577 6771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:46:21.491693 6771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:46:21.491778 6771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:46:21.491858 6771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:46:21.502363 6771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40137
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:46:21.503695 6771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:46:21.505045 6771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:46:21.517777 6788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:21.517747 6783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:21.517680 6786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:46:21.518621 6771 server_base.cc:1034] running on GCE node
I20260811 18:46:21.519408 6771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:46:21.520787 6771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:46:21.521998 6771 hybrid_clock.cc:648] HybridClock initialized: now 1786473981521944 us; error 52 us; skew 500 ppm
I20260811 18:46:21.522404 6771 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:46:21 dist-test-slave-z8x9 krb5kdc[6745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473981, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:46:21.525740 6771 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:46:21.527132 6771 webserver.cc:460] Webserver started at http://127.0.0.1:36891/ using document root <none> and password file <none>
I20260811 18:46:21.527797 6771 fs_manager.cc:359] Metadata directory not provided
I20260811 18:46:21.527900 6771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:46:21.528153 6771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:46:21.531155 6771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "b57b183e629340f29de44df9bb171983"
format_stamp: "Formatted at 2026-08-11 18:46:21 on dist-test-slave-z8x9"
server_key: "07d9b0203088aaf81b839b69c96886f2"
server_key_iv: "cd40a7d84887a9d01bd3fc9ca0aca617"
server_key_version: "encryptionkey@0"
I20260811 18:46:21.531894 6771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "b57b183e629340f29de44df9bb171983"
format_stamp: "Formatted at 2026-08-11 18:46:21 on dist-test-slave-z8x9"
server_key: "07d9b0203088aaf81b839b69c96886f2"
server_key_iv: "cd40a7d84887a9d01bd3fc9ca0aca617"
server_key_version: "encryptionkey@0"
I20260811 18:46:21.537303 6771 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 18:46:21.540835 6805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:46:21.542376 6771 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:46:21.542594 6771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "b57b183e629340f29de44df9bb171983"
format_stamp: "Formatted at 2026-08-11 18:46:21 on dist-test-slave-z8x9"
server_key: "07d9b0203088aaf81b839b69c96886f2"
server_key_iv: "cd40a7d84887a9d01bd3fc9ca0aca617"
server_key_version: "encryptionkey@0"
I20260811 18:46:21.542830 6771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:46:21.548816 6771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:46:21.553562 6771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:46:21.553887 6771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:46:21.572341 6771 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40137
I20260811 18:46:21.572386 6900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40137 every 8 connection(s)
I20260811 18:46:21.573837 6771 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:46:21.578058 6901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:46:21.578063 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 6771
I20260811 18:46:21.578261 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:46:21.578713 31462 external_mini_cluster.cc:1428] Setting key 2df39a0a1aa280d231a9b143e342acd8
I20260811 18:46:21.587160 6912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:21.580570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:54711 (local address 127.0.0.1:40137)
0811 18:46:21.581509 (+ 939us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:21.581523 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:21.581571 (+ 48us) server_negotiation.cc:407] Connection header received
0811 18:46:21.583184 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:21.583212 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:21.583659 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:21.584089 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:21.585196 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:21.585205 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:21.585222 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:46:21.585241 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:21.585313 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:46:21.585351 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:21.585569 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:21.585718 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 18:46:21.585920 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":369,"threads_started":1}
I20260811 18:46:21.590802 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:46:21.593453 6901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b57b183e629340f29de44df9bb171983: Bootstrap starting.
I20260811 18:46:21.597669 6901 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b57b183e629340f29de44df9bb171983: Neither blocks nor log segments found. Creating new log.
I20260811 18:46:21.598969 6912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:21.596314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:47723 (local address 127.0.0.1:40137)
0811 18:46:21.596488 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:21.596494 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:21.596872 (+ 378us) server_negotiation.cc:407] Connection header received
0811 18:46:21.597012 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:21.597017 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:21.597081 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:21.597231 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:21.598288 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:21.598294 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:21.598299 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:46:21.598317 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:21.598363 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:46:21.598383 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:21.598503 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:21.598582 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 18:46:21.598663 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 18:46:21.599017 6901 log.cc:824] T 00000000000000000000000000000000 P b57b183e629340f29de44df9bb171983: Log is configured to *not* fsync() on all Append() calls
I20260811 18:46:21.601902 6901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b57b183e629340f29de44df9bb171983: No bootstrap required, opened a new log
I20260811 18:46:21.609140 6901 raft_consensus.cc:348] T 00000000000000000000000000000000 P b57b183e629340f29de44df9bb171983 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b57b183e629340f29de44df9bb171983" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40137 } }
I20260811 18:46:21.609611 6901 raft_consensus.cc:374] T 00000000000000000000000000000000 P b57b183e629340f29de44df9bb171983 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:46:21.609683 6901 raft_consensus.cc:729] T 00000000000000000000000000000000 P b57b183e629340f29de44df9bb171983 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b57b183e629340f29de44df9bb171983, State: Initialized, Role: FOLLOWER
I20260811 18:46:21.610253 6901 consensus_queue.cc:260] T 00000000000000000000000000000000 P b57b183e629340f29de44df9bb171983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b57b183e629340f29de44df9bb171983" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40137 } }
I20260811 18:46:21.610512 6901 raft_consensus.cc:388] T 00000000000000000000000000000000 P b57b183e629340f29de44df9bb171983 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:46:21.610693 6901 raft_consensus.cc:482] T 00000000000000000000000000000000 P b57b183e629340f29de44df9bb171983 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:46:21.610872 6901 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b57b183e629340f29de44df9bb171983 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:46:21.613046 6901 raft_consensus.cc:504] T 00000000000000000000000000000000 P b57b183e629340f29de44df9bb171983 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b57b183e629340f29de44df9bb171983" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40137 } }
I20260811 18:46:21.613616 6901 leader_election.cc:302] T 00000000000000000000000000000000 P b57b183e629340f29de44df9bb171983 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b57b183e629340f29de44df9bb171983; no voters:
I20260811 18:46:21.614213 6901 leader_election.cc:288] T 00000000000000000000000000000000 P b57b183e629340f29de44df9bb171983 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:46:21.614394 6920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b57b183e629340f29de44df9bb171983 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:46:21.615073 6920 raft_consensus.cc:686] T 00000000000000000000000000000000 P b57b183e629340f29de44df9bb171983 [term 1 LEADER]: Becoming Leader. State: Replica: b57b183e629340f29de44df9bb171983, State: Running, Role: LEADER
I20260811 18:46:21.615664 6920 consensus_queue.cc:237] T 00000000000000000000000000000000 P b57b183e629340f29de44df9bb171983 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b57b183e629340f29de44df9bb171983" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40137 } }
I20260811 18:46:21.616403 6901 sys_catalog.cc:564] T 00000000000000000000000000000000 P b57b183e629340f29de44df9bb171983 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:46:21.618793 6923 sys_catalog.cc:455] T 00000000000000000000000000000000 P b57b183e629340f29de44df9bb171983 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b57b183e629340f29de44df9bb171983" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b57b183e629340f29de44df9bb171983" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40137 } } }
I20260811 18:46:21.619174 6923 sys_catalog.cc:458] T 00000000000000000000000000000000 P b57b183e629340f29de44df9bb171983 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:21.619906 6925 sys_catalog.cc:455] T 00000000000000000000000000000000 P b57b183e629340f29de44df9bb171983 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b57b183e629340f29de44df9bb171983. Latest consensus state: current_term: 1 leader_uuid: "b57b183e629340f29de44df9bb171983" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b57b183e629340f29de44df9bb171983" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40137 } } }
I20260811 18:46:21.620275 6925 sys_catalog.cc:458] T 00000000000000000000000000000000 P b57b183e629340f29de44df9bb171983 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:21.622080 6938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:46:21.627322 6938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:46:21.636726 6938 catalog_manager.cc:1347] Generated new cluster ID: 6eacc2c99a614af9a3e78c80a332e39b
I20260811 18:46:21.636852 6938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:46:21.654448 6938 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:46:21.654997 6938 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:46:21.662312 6938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b57b183e629340f29de44df9bb171983: Generated new TSK 0
I20260811 18:46:21.663194 6938 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:46:21.667670 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 6771
W20260811 18:46:21.668936 6892 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.69:54711
W20260811 18:46:21.669727 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 6771: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:46:21.669894 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 6771
2026-08-11T18:46:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2335 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:46:21 dist-test-slave-z8x9 krb5kdc[6964](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:46:21 dist-test-slave-z8x9 krb5kdc[6964](info): set up 2 sockets
Aug 11 18:46:21 dist-test-slave-z8x9 krb5kdc[6964](info): commencing operation
krb5kdc: starting...
W20260811 18:46:23.100731 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.381s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:46:23 dist-test-slave-z8x9 krb5kdc[6964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:46:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:46:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:46:23.469012 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:40051
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:42067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:40051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:46:23.822924 6986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:46:23.823232 6986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:46:23.823462 6986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:46:23.830845 6986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:46:23.830982 6986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:46:23.831079 6986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:46:23.831204 6986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:46:23.831274 6986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:46:23.841639 6986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:42067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:40051
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:40051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:46:23.843046 6986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:46:23.844477 6986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:46:23.857095 6998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:23.857069 7002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:23.857034 6999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:46:23.858424 6986 server_base.cc:1034] running on GCE node
I20260811 18:46:23.859266 6986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:46:23.860778 6986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:46:23.862017 6986 hybrid_clock.cc:648] HybridClock initialized: now 1786473983861993 us; error 59 us; skew 500 ppm
I20260811 18:46:23.862403 6986 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:46:23 dist-test-slave-z8x9 krb5kdc[6964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473983, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:46:23.865760 6986 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:46:23.867360 6986 webserver.cc:460] Webserver started at http://127.30.185.190:42839/ using document root <none> and password file <none>
I20260811 18:46:23.868016 6986 fs_manager.cc:359] Metadata directory not provided
I20260811 18:46:23.868124 6986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:46:23.868433 6986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:46:23.871529 6986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "6f2159fbad0b435fb84bc3822684a038"
format_stamp: "Formatted at 2026-08-11 18:46:23 on dist-test-slave-z8x9"
server_key: "93dddb8314303914a81fd856ab5e536e"
server_key_iv: "231da61bc84b3a988804d1eae3b0fa5f"
server_key_version: "encryptionkey@0"
I20260811 18:46:23.872262 6986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "6f2159fbad0b435fb84bc3822684a038"
format_stamp: "Formatted at 2026-08-11 18:46:23 on dist-test-slave-z8x9"
server_key: "93dddb8314303914a81fd856ab5e536e"
server_key_iv: "231da61bc84b3a988804d1eae3b0fa5f"
server_key_version: "encryptionkey@0"
I20260811 18:46:23.877611 6986 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 18:46:23.881088 7015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:46:23.882578 6986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:46:23.882730 6986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "6f2159fbad0b435fb84bc3822684a038"
format_stamp: "Formatted at 2026-08-11 18:46:23 on dist-test-slave-z8x9"
server_key: "93dddb8314303914a81fd856ab5e536e"
server_key_iv: "231da61bc84b3a988804d1eae3b0fa5f"
server_key_version: "encryptionkey@0"
I20260811 18:46:23.882896 6986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:46:23.907675 6986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:46:23.912678 6986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:46:23.912947 6986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:46:23.934477 6986 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:40051
I20260811 18:46:23.934509 7107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:40051 every 8 connection(s)
I20260811 18:46:23.936081 6986 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:46:23.940116 7108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:46:23.945730 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 6986
I20260811 18:46:23.945952 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:46:23.946434 31462 external_mini_cluster.cc:1428] Setting key b9f7f1a93e1a133e8235f27c81747944
I20260811 18:46:23.952142 7108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2159fbad0b435fb84bc3822684a038: Bootstrap starting.
I20260811 18:46:23.956285 7108 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f2159fbad0b435fb84bc3822684a038: Neither blocks nor log segments found. Creating new log.
I20260811 18:46:23.957749 7108 log.cc:824] T 00000000000000000000000000000000 P 6f2159fbad0b435fb84bc3822684a038: Log is configured to *not* fsync() on all Append() calls
I20260811 18:46:23.961755 7108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f2159fbad0b435fb84bc3822684a038: No bootstrap required, opened a new log
Aug 11 18:46:23 dist-test-slave-z8x9 krb5kdc[6964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:46:23.970888 7108 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f2159fbad0b435fb84bc3822684a038 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2159fbad0b435fb84bc3822684a038" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40051 } }
I20260811 18:46:23.971367 7108 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f2159fbad0b435fb84bc3822684a038 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:46:23.971508 7108 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f2159fbad0b435fb84bc3822684a038 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f2159fbad0b435fb84bc3822684a038, State: Initialized, Role: FOLLOWER
I20260811 18:46:23.972295 7108 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f2159fbad0b435fb84bc3822684a038 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2159fbad0b435fb84bc3822684a038" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40051 } }
I20260811 18:46:23.972620 7108 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f2159fbad0b435fb84bc3822684a038 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:46:23.972795 7108 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f2159fbad0b435fb84bc3822684a038 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:46:23.973033 7108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f2159fbad0b435fb84bc3822684a038 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:46:23.973726 7113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:23.948344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47429 (local address 127.30.185.190:40051)
0811 18:46:23.949317 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:23.949330 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:23.949380 (+ 50us) server_negotiation.cc:407] Connection header received
0811 18:46:23.951666 (+ 2286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:23.951703 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:23.952139 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:23.952541 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:23.954575 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:23.956047 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:23.958093 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:23.959308 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:23.959477 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:23.964844 (+ 5367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:23.964852 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:23.964868 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:23.964909 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:23.968557 (+ 3648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:23.969762 (+ 1205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:23.969770 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:23.969781 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:23.969977 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:23.970514 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:23.970518 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:23.970520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:23.971109 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:23.971480 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:23.972021 (+ 541us) server_negotiation.cc:299] Negotiation successful
0811 18:46:23.972366 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":415,"threads_started":1}
I20260811 18:46:23.975765 7108 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f2159fbad0b435fb84bc3822684a038 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2159fbad0b435fb84bc3822684a038" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40051 } }
I20260811 18:46:23.976328 7108 leader_election.cc:302] T 00000000000000000000000000000000 P 6f2159fbad0b435fb84bc3822684a038 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f2159fbad0b435fb84bc3822684a038; no voters:
I20260811 18:46:23.977001 7108 leader_election.cc:288] T 00000000000000000000000000000000 P 6f2159fbad0b435fb84bc3822684a038 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:46:23.977109 7115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f2159fbad0b435fb84bc3822684a038 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:46:23.978724 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:46:23.979626 7108 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f2159fbad0b435fb84bc3822684a038 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:46:23.984265 7115 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f2159fbad0b435fb84bc3822684a038 [term 1 LEADER]: Becoming Leader. State: Replica: 6f2159fbad0b435fb84bc3822684a038, State: Running, Role: LEADER
I20260811 18:46:23.984988 7115 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f2159fbad0b435fb84bc3822684a038 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2159fbad0b435fb84bc3822684a038" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40051 } }
I20260811 18:46:23.988377 7121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2159fbad0b435fb84bc3822684a038 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f2159fbad0b435fb84bc3822684a038. Latest consensus state: current_term: 1 leader_uuid: "6f2159fbad0b435fb84bc3822684a038" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2159fbad0b435fb84bc3822684a038" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40051 } } }
I20260811 18:46:23.988801 7121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2159fbad0b435fb84bc3822684a038 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:23.988350 7118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f2159fbad0b435fb84bc3822684a038 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f2159fbad0b435fb84bc3822684a038" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f2159fbad0b435fb84bc3822684a038" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 40051 } } }
I20260811 18:46:23.989305 7118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f2159fbad0b435fb84bc3822684a038 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:23.997778 7134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:46:24.004482 7134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:46:24.016052 7113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:23.995969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58945 (local address 127.30.185.190:40051)
0811 18:46:23.996136 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:23.996141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:23.996893 (+ 752us) server_negotiation.cc:407] Connection header received
0811 18:46:23.997268 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:23.997275 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:23.997360 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:23.997441 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:23.999221 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:24.000552 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:24.002421 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:24.003143 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:24.003279 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:24.006921 (+ 3642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:24.006928 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:24.006932 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:24.006973 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:24.010773 (+ 3800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:24.011764 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:24.011772 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:24.011776 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:24.011928 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:24.012547 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:24.012555 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:24.012560 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:24.013212 (+ 652us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:24.013584 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:24.015239 (+ 1655us) server_negotiation.cc:299] Negotiation successful
0811 18:46:24.015501 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 18:46:24.024190 7134 catalog_manager.cc:1347] Generated new cluster ID: fc14a5fce3444363b47be9afaa53d1a5
I20260811 18:46:24.024591 7134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:46:24.050868 7134 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:46:24.052773 7134 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:46:24.063074 7134 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f2159fbad0b435fb84bc3822684a038: Generated new TSK 0
I20260811 18:46:24.064281 7134 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:46:24.071730 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 6986
I20260811 18:46:24.073614 7094 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:46:24.166919 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 6986
W20260811 18:46:24.190606 6962 connection.cc:535] client connection to 127.30.185.190:40051 recv error: Network error: failed to read from TLS socket (remote: 127.30.185.190:40051): Connection reset by peer (error 104)
2026-08-11T18:46:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2489 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:46:24 dist-test-slave-z8x9 krb5kdc[7166](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:46:24 dist-test-slave-z8x9 krb5kdc[7166](info): set up 2 sockets
Aug 11 18:46:24 dist-test-slave-z8x9 krb5kdc[7166](info): commencing operation
krb5kdc: starting...
W20260811 18:46:25.585044 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.375s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:46:25 dist-test-slave-z8x9 krb5kdc[7166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:46:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:46:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:46:25.948797 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:39321
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:43027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:39321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:46:26.305691 7188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:46:26.306015 7188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:46:26.306130 7188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:46:26.313657 7188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:46:26.313777 7188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:46:26.313824 7188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:46:26.313886 7188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:46:26.313931 7188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:46:26.323901 7188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:43027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:39321
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:39321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:46:26.325225 7188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:46:26.326560 7188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:46:26.338373 7197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:26.338493 7202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:26.338505 7204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:46:26.340049 7188 server_base.cc:1034] running on GCE node
I20260811 18:46:26.340833 7188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:46:26.342285 7188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:46:26.343542 7188 hybrid_clock.cc:648] HybridClock initialized: now 1786473986343490 us; error 30 us; skew 500 ppm
I20260811 18:46:26.343927 7188 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:46:26 dist-test-slave-z8x9 krb5kdc[7166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473986, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:46:26.350781 7188 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:46:26.352319 7188 webserver.cc:460] Webserver started at http://127.30.185.190:45971/ using document root <none> and password file <none>
I20260811 18:46:26.353040 7188 fs_manager.cc:359] Metadata directory not provided
I20260811 18:46:26.353230 7188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:46:26.353560 7188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:46:26.356571 7188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "41013fe08b4547c6bd4e864422211553"
format_stamp: "Formatted at 2026-08-11 18:46:26 on dist-test-slave-z8x9"
server_key: "b1350dc31fd7cc72847aa7ca45331935"
server_key_iv: "b017b5d4f8710295413cafed1f7fa2a7"
server_key_version: "encryptionkey@0"
I20260811 18:46:26.357448 7188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "41013fe08b4547c6bd4e864422211553"
format_stamp: "Formatted at 2026-08-11 18:46:26 on dist-test-slave-z8x9"
server_key: "b1350dc31fd7cc72847aa7ca45331935"
server_key_iv: "b017b5d4f8710295413cafed1f7fa2a7"
server_key_version: "encryptionkey@0"
I20260811 18:46:26.362845 7188 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 18:46:26.366526 7221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:46:26.367969 7188 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:46:26.368163 7188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "41013fe08b4547c6bd4e864422211553"
format_stamp: "Formatted at 2026-08-11 18:46:26 on dist-test-slave-z8x9"
server_key: "b1350dc31fd7cc72847aa7ca45331935"
server_key_iv: "b017b5d4f8710295413cafed1f7fa2a7"
server_key_version: "encryptionkey@0"
I20260811 18:46:26.368389 7188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:46:26.396308 7188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:46:26.400735 7188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:46:26.401068 7188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:46:26.420796 7188 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:39321
I20260811 18:46:26.420820 7326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:39321 every 8 connection(s)
I20260811 18:46:26.422302 7188 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:46:26.425390 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 7188
I20260811 18:46:26.425583 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:46:26.426102 31462 external_mini_cluster.cc:1428] Setting key 9b1f27e935fde658ae508de06f19331f
I20260811 18:46:26.426622 7327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:46:26.440569 7333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:26.428049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35187 (local address 127.30.185.190:39321)
0811 18:46:26.428881 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:26.428892 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:26.428933 (+ 41us) server_negotiation.cc:407] Connection header received
0811 18:46:26.431340 (+ 2407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:26.431363 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:26.431775 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:26.432123 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:26.432799 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:26.434216 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:26.436023 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:26.436687 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:26.436842 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:26.438258 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:26.438266 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:26.438279 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:46:26.438866 (+ 587us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:26.438948 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:46:26.438973 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:26.439098 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:26.439366 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 18:46:26.439560 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":388,"threads_started":1}
I20260811 18:46:26.441028 7327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41013fe08b4547c6bd4e864422211553: Bootstrap starting.
I20260811 18:46:26.444763 7327 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41013fe08b4547c6bd4e864422211553: Neither blocks nor log segments found. Creating new log.
I20260811 18:46:26.444875 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:46:26.445889 7327 log.cc:824] T 00000000000000000000000000000000 P 41013fe08b4547c6bd4e864422211553: Log is configured to *not* fsync() on all Append() calls
I20260811 18:46:26.448684 7327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41013fe08b4547c6bd4e864422211553: No bootstrap required, opened a new log
I20260811 18:46:26.456637 7333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:26.449634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48261 (local address 127.30.185.190:39321)
0811 18:46:26.449821 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:26.449827 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:26.450270 (+ 443us) server_negotiation.cc:407] Connection header received
0811 18:46:26.450423 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:26.450428 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:26.450510 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:26.450625 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:26.451121 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:26.452323 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:26.453736 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:26.454503 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:26.454651 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:26.455811 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:26.455818 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:26.455823 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:46:26.455845 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:26.455899 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:46:26.455921 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:26.456057 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:26.456213 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 18:46:26.456275 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 18:46:26.459690 7327 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41013fe08b4547c6bd4e864422211553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41013fe08b4547c6bd4e864422211553" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39321 } }
I20260811 18:46:26.460112 7327 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41013fe08b4547c6bd4e864422211553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:46:26.460237 7327 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41013fe08b4547c6bd4e864422211553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41013fe08b4547c6bd4e864422211553, State: Initialized, Role: FOLLOWER
I20260811 18:46:26.460888 7327 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41013fe08b4547c6bd4e864422211553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41013fe08b4547c6bd4e864422211553" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39321 } }
I20260811 18:46:26.461153 7327 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41013fe08b4547c6bd4e864422211553 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:46:26.461376 7327 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41013fe08b4547c6bd4e864422211553 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:46:26.461584 7327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41013fe08b4547c6bd4e864422211553 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:46:26.465081 7327 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41013fe08b4547c6bd4e864422211553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41013fe08b4547c6bd4e864422211553" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39321 } }
I20260811 18:46:26.465808 7327 leader_election.cc:302] T 00000000000000000000000000000000 P 41013fe08b4547c6bd4e864422211553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41013fe08b4547c6bd4e864422211553; no voters:
I20260811 18:46:26.466585 7327 leader_election.cc:288] T 00000000000000000000000000000000 P 41013fe08b4547c6bd4e864422211553 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:46:26.466684 7347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41013fe08b4547c6bd4e864422211553 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:46:26.472673 7347 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41013fe08b4547c6bd4e864422211553 [term 1 LEADER]: Becoming Leader. State: Replica: 41013fe08b4547c6bd4e864422211553, State: Running, Role: LEADER
I20260811 18:46:26.473407 7347 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41013fe08b4547c6bd4e864422211553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41013fe08b4547c6bd4e864422211553" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39321 } }
I20260811 18:46:26.473814 7327 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41013fe08b4547c6bd4e864422211553 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:46:26.476719 7351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41013fe08b4547c6bd4e864422211553 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41013fe08b4547c6bd4e864422211553" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41013fe08b4547c6bd4e864422211553" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39321 } } }
I20260811 18:46:26.477280 7351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41013fe08b4547c6bd4e864422211553 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:26.478907 7357 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:46:26.479516 7348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41013fe08b4547c6bd4e864422211553 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41013fe08b4547c6bd4e864422211553. Latest consensus state: current_term: 1 leader_uuid: "41013fe08b4547c6bd4e864422211553" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41013fe08b4547c6bd4e864422211553" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 39321 } } }
I20260811 18:46:26.479914 7348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41013fe08b4547c6bd4e864422211553 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:26.485035 7357 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:46:26.498421 7357 catalog_manager.cc:1347] Generated new cluster ID: 429b43104ce843ef941a9d914fbc2d7e
I20260811 18:46:26.498551 7357 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:46:26.544147 7357 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:46:26.546168 7357 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:46:26.570490 7357 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41013fe08b4547c6bd4e864422211553: Generated new TSK 0
I20260811 18:46:26.571398 7357 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:46:26.592410 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 7188
W20260811 18:46:26.593647 7317 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35187
W20260811 18:46:26.594465 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 7188: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:46:26.594630 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 7188
W20260811 18:46:26.624253 7164 connection.cc:535] client connection to 127.30.185.190:39321 recv error: Network error: failed to read from TLS socket (remote: 127.30.185.190:39321): Connection reset by peer (error 104)
2026-08-11T18:46:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2433 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:46:26 dist-test-slave-z8x9 krb5kdc[7384](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:46:26 dist-test-slave-z8x9 krb5kdc[7384](info): set up 2 sockets
Aug 11 18:46:26 dist-test-slave-z8x9 krb5kdc[7384](info): commencing operation
krb5kdc: starting...
W20260811 18:46:28.020731 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.377s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:46:28 dist-test-slave-z8x9 krb5kdc[7384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:46:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:46:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:46:28.394609 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:44531
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:38891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:44531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:46:28.924605 7410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:46:28.925119 7410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:46:28.925483 7410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:46:28.937403 7410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:46:28.937604 7410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:46:28.937786 7410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:46:28.938081 7410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:46:28.938227 7410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:46:28.954227 7410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:38891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:44531
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:44531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:46:28.956501 7410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:46:28.959020 7410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:46:28.979885 7416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:28.979985 7420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:28.979965 7417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:46:28.983289 7410 server_base.cc:1034] running on GCE node
I20260811 18:46:28.985078 7410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:46:28.987821 7410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:46:28.989151 7410 hybrid_clock.cc:648] HybridClock initialized: now 1786473988989089 us; error 53 us; skew 500 ppm
I20260811 18:46:28.989868 7410 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:46:28 dist-test-slave-z8x9 krb5kdc[7384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473988, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:46:28.995504 7410 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:46:28.998467 7410 webserver.cc:460] Webserver started at http://127.30.185.190:41183/ using document root <none> and password file <none>
I20260811 18:46:28.999725 7410 fs_manager.cc:359] Metadata directory not provided
I20260811 18:46:28.999954 7410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:46:29.000644 7410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:46:29.006387 7410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "9b4caea8c0a7484f8d616ab8e3683ac3"
format_stamp: "Formatted at 2026-08-11 18:46:28 on dist-test-slave-z8x9"
server_key: "e8cfb22aa87c73bd904ef6515c2d51ae"
server_key_iv: "a986015d992069425a6d095059996292"
server_key_version: "encryptionkey@0"
I20260811 18:46:29.007944 7410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "9b4caea8c0a7484f8d616ab8e3683ac3"
format_stamp: "Formatted at 2026-08-11 18:46:28 on dist-test-slave-z8x9"
server_key: "e8cfb22aa87c73bd904ef6515c2d51ae"
server_key_iv: "a986015d992069425a6d095059996292"
server_key_version: "encryptionkey@0"
I20260811 18:46:29.018481 7410 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.010s sys 0.001s
I20260811 18:46:29.025022 7430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:46:29.027485 7410 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.003s
I20260811 18:46:29.027837 7410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "9b4caea8c0a7484f8d616ab8e3683ac3"
format_stamp: "Formatted at 2026-08-11 18:46:28 on dist-test-slave-z8x9"
server_key: "e8cfb22aa87c73bd904ef6515c2d51ae"
server_key_iv: "a986015d992069425a6d095059996292"
server_key_version: "encryptionkey@0"
I20260811 18:46:29.028208 7410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:46:29.039501 7410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:46:29.047639 7410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:46:29.048136 7410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:46:29.081434 7410 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:44531
I20260811 18:46:29.081570 7520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:44531 every 8 connection(s)
I20260811 18:46:29.083892 7410 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:46:29.089397 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 7410
I20260811 18:46:29.089718 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:46:29.090818 31462 external_mini_cluster.cc:1428] Setting key c2e5980082565997ba64dc7b76077b84
I20260811 18:46:29.090970 7521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:46:29.105708 7537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:29.094054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35809 (local address 127.30.185.190:44531)
0811 18:46:29.095484 (+ 1430us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:29.095547 (+ 63us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:29.095640 (+ 93us) server_negotiation.cc:407] Connection header received
0811 18:46:29.099082 (+ 3442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:29.099127 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:29.099764 (+ 637us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:29.100383 (+ 619us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:29.102575 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:29.102592 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:29.102617 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:46:29.102647 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:29.102777 (+ 130us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:46:29.102835 (+ 58us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:29.103130 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:29.103391 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 18:46:29.103824 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1049,"thread_start_us":702,"threads_started":1}
I20260811 18:46:29.112270 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:46:29.114845 7521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b4caea8c0a7484f8d616ab8e3683ac3: Bootstrap starting.
I20260811 18:46:29.120441 7521 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b4caea8c0a7484f8d616ab8e3683ac3: Neither blocks nor log segments found. Creating new log.
I20260811 18:46:29.122417 7521 log.cc:824] T 00000000000000000000000000000000 P 9b4caea8c0a7484f8d616ab8e3683ac3: Log is configured to *not* fsync() on all Append() calls
I20260811 18:46:29.123665 7537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:29.119605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53671 (local address 127.30.185.190:44531)
0811 18:46:29.119885 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:29.119898 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:29.120513 (+ 615us) server_negotiation.cc:407] Connection header received
0811 18:46:29.120768 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:29.120792 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:29.120909 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:29.121092 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:29.122377 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:29.122398 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:29.122407 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:46:29.122450 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:29.122545 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:46:29.122604 (+ 59us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:29.122812 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:29.122975 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 18:46:29.123067 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 18:46:29.128093 7521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b4caea8c0a7484f8d616ab8e3683ac3: No bootstrap required, opened a new log
I20260811 18:46:29.142930 7521 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b4caea8c0a7484f8d616ab8e3683ac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b4caea8c0a7484f8d616ab8e3683ac3" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44531 } }
I20260811 18:46:29.143780 7521 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b4caea8c0a7484f8d616ab8e3683ac3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:46:29.143903 7521 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b4caea8c0a7484f8d616ab8e3683ac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b4caea8c0a7484f8d616ab8e3683ac3, State: Initialized, Role: FOLLOWER
I20260811 18:46:29.144558 7521 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b4caea8c0a7484f8d616ab8e3683ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b4caea8c0a7484f8d616ab8e3683ac3" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44531 } }
I20260811 18:46:29.144887 7521 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b4caea8c0a7484f8d616ab8e3683ac3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:46:29.145043 7521 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b4caea8c0a7484f8d616ab8e3683ac3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:46:29.145318 7521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b4caea8c0a7484f8d616ab8e3683ac3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:46:29.148874 7521 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b4caea8c0a7484f8d616ab8e3683ac3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b4caea8c0a7484f8d616ab8e3683ac3" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44531 } }
I20260811 18:46:29.149711 7521 leader_election.cc:302] T 00000000000000000000000000000000 P 9b4caea8c0a7484f8d616ab8e3683ac3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b4caea8c0a7484f8d616ab8e3683ac3; no voters:
I20260811 18:46:29.150609 7521 leader_election.cc:288] T 00000000000000000000000000000000 P 9b4caea8c0a7484f8d616ab8e3683ac3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:46:29.150803 7546 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b4caea8c0a7484f8d616ab8e3683ac3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:46:29.153353 7546 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b4caea8c0a7484f8d616ab8e3683ac3 [term 1 LEADER]: Becoming Leader. State: Replica: 9b4caea8c0a7484f8d616ab8e3683ac3, State: Running, Role: LEADER
I20260811 18:46:29.154292 7546 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b4caea8c0a7484f8d616ab8e3683ac3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b4caea8c0a7484f8d616ab8e3683ac3" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44531 } }
I20260811 18:46:29.155077 7521 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b4caea8c0a7484f8d616ab8e3683ac3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:46:29.159685 7549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b4caea8c0a7484f8d616ab8e3683ac3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b4caea8c0a7484f8d616ab8e3683ac3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b4caea8c0a7484f8d616ab8e3683ac3" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44531 } } }
I20260811 18:46:29.159585 7555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b4caea8c0a7484f8d616ab8e3683ac3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b4caea8c0a7484f8d616ab8e3683ac3. Latest consensus state: current_term: 1 leader_uuid: "9b4caea8c0a7484f8d616ab8e3683ac3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b4caea8c0a7484f8d616ab8e3683ac3" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 44531 } } }
I20260811 18:46:29.160460 7549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b4caea8c0a7484f8d616ab8e3683ac3 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:29.160493 7555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b4caea8c0a7484f8d616ab8e3683ac3 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:29.162159 7564 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:46:29.172197 7564 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:46:29.187798 7564 catalog_manager.cc:1347] Generated new cluster ID: c3e85b1eafa44202b3aabe799c9d0d3b
I20260811 18:46:29.188160 7564 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:46:29.215284 7564 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:46:29.216418 7564 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:46:29.237617 7564 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b4caea8c0a7484f8d616ab8e3683ac3: Generated new TSK 0
I20260811 18:46:29.239153 7564 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:46:29.247737 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 7410
W20260811 18:46:29.249780 7515 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35809
W20260811 18:46:29.251061 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 7410: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:46:29.251381 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 7410
2026-08-11T18:46:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2686 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:46:29 dist-test-slave-z8x9 krb5kdc[7587](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:46:29 dist-test-slave-z8x9 krb5kdc[7587](info): set up 2 sockets
Aug 11 18:46:29 dist-test-slave-z8x9 krb5kdc[7587](info): commencing operation
krb5kdc: starting...
W20260811 18:46:30.977121 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.644s user 0.004s sys 0.079s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:46:31 dist-test-slave-z8x9 krb5kdc[7587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:46:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:46:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:46:31.344916 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:45277
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:46127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:45277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:46:31.707460 7617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:46:31.707775 7617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:46:31.707949 7617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:46:31.715437 7617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:46:31.715564 7617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:46:31.715687 7617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:46:31.715792 7617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:46:31.715853 7617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:46:31.726308 7617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:46127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:45277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:45277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:46:31.727720 7617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:46:31.729115 7617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:46:31.741779 7629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:31.741849 7634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:31.741825 7632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:46:31.742792 7617 server_base.cc:1034] running on GCE node
I20260811 18:46:31.743700 7617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:46:31.745373 7617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:46:31.746599 7617 hybrid_clock.cc:648] HybridClock initialized: now 1786473991746543 us; error 63 us; skew 500 ppm
I20260811 18:46:31.746994 7617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:46:31 dist-test-slave-z8x9 krb5kdc[7587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473991, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:46:31.754786 7617 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:46:31.756536 7617 webserver.cc:460] Webserver started at http://127.30.185.190:42339/ using document root <none> and password file <none>
I20260811 18:46:31.757365 7617 fs_manager.cc:359] Metadata directory not provided
I20260811 18:46:31.757529 7617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:46:31.757856 7617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:46:31.761013 7617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "84307b9277f84804b504890c678cb900"
format_stamp: "Formatted at 2026-08-11 18:46:31 on dist-test-slave-z8x9"
server_key: "a0ef97de6643ddc84884d8930d27e322"
server_key_iv: "d4cf2d8648aef5e2603cd3db5122bfad"
server_key_version: "encryptionkey@0"
I20260811 18:46:31.761897 7617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "84307b9277f84804b504890c678cb900"
format_stamp: "Formatted at 2026-08-11 18:46:31 on dist-test-slave-z8x9"
server_key: "a0ef97de6643ddc84884d8930d27e322"
server_key_iv: "d4cf2d8648aef5e2603cd3db5122bfad"
server_key_version: "encryptionkey@0"
I20260811 18:46:31.767259 7617 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:46:31.771013 7646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:46:31.772430 7617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:46:31.772631 7617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "84307b9277f84804b504890c678cb900"
format_stamp: "Formatted at 2026-08-11 18:46:31 on dist-test-slave-z8x9"
server_key: "a0ef97de6643ddc84884d8930d27e322"
server_key_iv: "d4cf2d8648aef5e2603cd3db5122bfad"
server_key_version: "encryptionkey@0"
I20260811 18:46:31.772845 7617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:46:31.787006 7617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:46:31.791330 7617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:46:31.791669 7617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:46:31.811264 7617 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:45277
I20260811 18:46:31.811319 7743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:45277 every 8 connection(s)
I20260811 18:46:31.812793 7617 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:46:31.816555 7745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:46:31.822137 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 7617
I20260811 18:46:31.822365 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:46:31.822841 31462 external_mini_cluster.cc:1428] Setting key 8ac5bdf44c69f7e262aef2b9270dc908
I20260811 18:46:31.828781 7745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84307b9277f84804b504890c678cb900: Bootstrap starting.
I20260811 18:46:31.832644 7745 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84307b9277f84804b504890c678cb900: Neither blocks nor log segments found. Creating new log.
I20260811 18:46:31.833892 7745 log.cc:824] T 00000000000000000000000000000000 P 84307b9277f84804b504890c678cb900: Log is configured to *not* fsync() on all Append() calls
I20260811 18:46:31.837023 7745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84307b9277f84804b504890c678cb900: No bootstrap required, opened a new log
Aug 11 18:46:31 dist-test-slave-z8x9 krb5kdc[7587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:46:31.845542 7745 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84307b9277f84804b504890c678cb900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84307b9277f84804b504890c678cb900" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 45277 } }
I20260811 18:46:31.845983 7745 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84307b9277f84804b504890c678cb900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:46:31.846149 7745 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84307b9277f84804b504890c678cb900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84307b9277f84804b504890c678cb900, State: Initialized, Role: FOLLOWER
I20260811 18:46:31.846761 7745 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84307b9277f84804b504890c678cb900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84307b9277f84804b504890c678cb900" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 45277 } }
I20260811 18:46:31.847039 7745 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84307b9277f84804b504890c678cb900 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:46:31.847203 7745 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84307b9277f84804b504890c678cb900 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:46:31.847394 7745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84307b9277f84804b504890c678cb900 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:46:31.849119 7750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:31.824853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47323 (local address 127.30.185.190:45277)
0811 18:46:31.825820 (+ 967us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:31.825836 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:31.825889 (+ 53us) server_negotiation.cc:407] Connection header received
0811 18:46:31.828391 (+ 2502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:31.828422 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:31.828948 (+ 526us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:31.829422 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:31.830856 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:31.832906 (+ 2050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:31.834408 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:31.835266 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:31.835475 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:31.840120 (+ 4645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:31.840132 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:31.840152 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:31.840200 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:31.843919 (+ 3719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:31.845105 (+ 1186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:31.845120 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:31.845126 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:31.845443 (+ 317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:31.845903 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:31.845909 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:31.845914 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:31.846728 (+ 814us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:31.847099 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:31.847544 (+ 445us) server_negotiation.cc:299] Negotiation successful
0811 18:46:31.848035 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":437,"threads_started":1}
I20260811 18:46:31.850327 7745 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84307b9277f84804b504890c678cb900 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84307b9277f84804b504890c678cb900" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 45277 } }
I20260811 18:46:31.850896 7745 leader_election.cc:302] T 00000000000000000000000000000000 P 84307b9277f84804b504890c678cb900 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84307b9277f84804b504890c678cb900; no voters:
I20260811 18:46:31.851536 7745 leader_election.cc:288] T 00000000000000000000000000000000 P 84307b9277f84804b504890c678cb900 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:46:31.851738 7753 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84307b9277f84804b504890c678cb900 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:46:31.853452 7753 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84307b9277f84804b504890c678cb900 [term 1 LEADER]: Becoming Leader. State: Replica: 84307b9277f84804b504890c678cb900, State: Running, Role: LEADER
I20260811 18:46:31.853708 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:46:31.854167 7745 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84307b9277f84804b504890c678cb900 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:46:31.854115 7753 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84307b9277f84804b504890c678cb900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84307b9277f84804b504890c678cb900" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 45277 } }
I20260811 18:46:31.864063 7755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84307b9277f84804b504890c678cb900 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84307b9277f84804b504890c678cb900" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84307b9277f84804b504890c678cb900" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 45277 } } }
I20260811 18:46:31.864627 7755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84307b9277f84804b504890c678cb900 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:31.865257 7756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84307b9277f84804b504890c678cb900 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84307b9277f84804b504890c678cb900. Latest consensus state: current_term: 1 leader_uuid: "84307b9277f84804b504890c678cb900" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84307b9277f84804b504890c678cb900" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 45277 } } }
I20260811 18:46:31.865613 7756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84307b9277f84804b504890c678cb900 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:31.868177 7780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:46:31.875442 7780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:46:31.888727 7750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:31.863801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37249 (local address 127.30.185.190:45277)
0811 18:46:31.867849 (+ 4048us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:31.867859 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:31.868574 (+ 715us) server_negotiation.cc:407] Connection header received
0811 18:46:31.868791 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:31.868796 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:31.868878 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:31.868988 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:31.871653 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:31.872877 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:31.874754 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:31.875467 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:31.875597 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:31.879288 (+ 3691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:31.879300 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:31.879307 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:31.879347 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:31.883376 (+ 4029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:31.884316 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:31.884321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:31.884325 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:31.884475 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:31.884943 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:31.884948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:31.884951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:31.885559 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:31.886045 (+ 486us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:31.887827 (+ 1782us) server_negotiation.cc:299] Negotiation successful
0811 18:46:31.888148 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3875}
I20260811 18:46:31.898650 7780 catalog_manager.cc:1347] Generated new cluster ID: 3331bb077716432096f102df5305b156
I20260811 18:46:31.898787 7780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:46:31.917680 7780 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:46:31.919080 7780 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:46:31.928107 7780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84307b9277f84804b504890c678cb900: Generated new TSK 0
I20260811 18:46:31.929095 7780 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:46:31.945510 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 7617
I20260811 18:46:31.946666 7735 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:46:32.042135 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 7617
2026-08-11T18:46:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2756 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:46:32 dist-test-slave-z8x9 krb5kdc[7802](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:46:32 dist-test-slave-z8x9 krb5kdc[7802](info): set up 2 sockets
Aug 11 18:46:32 dist-test-slave-z8x9 krb5kdc[7802](info): commencing operation
krb5kdc: starting...
W20260811 18:46:33.465504 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.378s user 0.000s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:46:33 dist-test-slave-z8x9 krb5kdc[7802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:46:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:46:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:46:33.839124 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:34883
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:41869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:34883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:46:34.192111 7828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:46:34.192430 7828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:46:34.192617 7828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:46:34.199973 7828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:46:34.200091 7828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:46:34.200217 7828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:46:34.200332 7828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:46:34.200402 7828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:46:34.210216 7828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:41869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:34883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:34883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:46:34.211521 7828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:46:34.212867 7828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:46:34.227022 7841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:34.226969 7838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:34.227042 7837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:46:34.227787 7828 server_base.cc:1034] running on GCE node
I20260811 18:46:34.228715 7828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:46:34.230348 7828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:46:34.231608 7828 hybrid_clock.cc:648] HybridClock initialized: now 1786473994231579 us; error 59 us; skew 500 ppm
I20260811 18:46:34.232071 7828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:46:34 dist-test-slave-z8x9 krb5kdc[7802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473994, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:46:34.235626 7828 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:46:34.237362 7828 webserver.cc:460] Webserver started at http://127.30.185.190:33901/ using document root <none> and password file <none>
I20260811 18:46:34.238088 7828 fs_manager.cc:359] Metadata directory not provided
I20260811 18:46:34.238247 7828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:46:34.238579 7828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:46:34.241879 7828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "7092a88ef90c42ffbf1205fac5711638"
format_stamp: "Formatted at 2026-08-11 18:46:34 on dist-test-slave-z8x9"
server_key: "e960281de9933dd5776ff94ff9041879"
server_key_iv: "e2bb23adf211785bb3757093de817d63"
server_key_version: "encryptionkey@0"
I20260811 18:46:34.242723 7828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "7092a88ef90c42ffbf1205fac5711638"
format_stamp: "Formatted at 2026-08-11 18:46:34 on dist-test-slave-z8x9"
server_key: "e960281de9933dd5776ff94ff9041879"
server_key_iv: "e2bb23adf211785bb3757093de817d63"
server_key_version: "encryptionkey@0"
I20260811 18:46:34.248260 7828 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 18:46:34.251967 7853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:46:34.253364 7828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:46:34.253562 7828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "7092a88ef90c42ffbf1205fac5711638"
format_stamp: "Formatted at 2026-08-11 18:46:34 on dist-test-slave-z8x9"
server_key: "e960281de9933dd5776ff94ff9041879"
server_key_iv: "e2bb23adf211785bb3757093de817d63"
server_key_version: "encryptionkey@0"
I20260811 18:46:34.253777 7828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:46:34.293260 7828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:46:34.298115 7828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:46:34.298460 7828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:46:34.318383 7828 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:34883
I20260811 18:46:34.318409 7975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:34883 every 8 connection(s)
I20260811 18:46:34.319808 7828 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:46:34.323556 7976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:46:34.326026 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 7828
I20260811 18:46:34.326218 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:46:34.326776 31462 external_mini_cluster.cc:1428] Setting key c34a0237c3b917ff5d45d365d32e3253
I20260811 18:46:34.336812 7976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7092a88ef90c42ffbf1205fac5711638: Bootstrap starting.
I20260811 18:46:34.340675 7976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7092a88ef90c42ffbf1205fac5711638: Neither blocks nor log segments found. Creating new log.
I20260811 18:46:34.340998 7981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:34.328731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56325 (local address 127.30.185.190:34883)
0811 18:46:34.329650 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:34.329663 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:34.329712 (+ 49us) server_negotiation.cc:407] Connection header received
0811 18:46:34.331477 (+ 1765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:34.331503 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:34.331931 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:34.332339 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:34.332973 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:34.334405 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:34.336267 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:34.337003 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:34.337155 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:34.338556 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:34.338567 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:34.338582 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:46:34.339166 (+ 584us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:34.339247 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:46:34.339273 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:34.339370 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:34.339612 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 18:46:34.339900 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":428,"threads_started":1}
I20260811 18:46:34.341935 7976 log.cc:824] T 00000000000000000000000000000000 P 7092a88ef90c42ffbf1205fac5711638: Log is configured to *not* fsync() on all Append() calls
I20260811 18:46:34.345216 7976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7092a88ef90c42ffbf1205fac5711638: No bootstrap required, opened a new log
I20260811 18:46:34.345252 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:46:34.356014 7976 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7092a88ef90c42ffbf1205fac5711638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7092a88ef90c42ffbf1205fac5711638" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 34883 } }
I20260811 18:46:34.356485 7976 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7092a88ef90c42ffbf1205fac5711638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:46:34.356611 7976 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7092a88ef90c42ffbf1205fac5711638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7092a88ef90c42ffbf1205fac5711638, State: Initialized, Role: FOLLOWER
I20260811 18:46:34.357437 7976 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7092a88ef90c42ffbf1205fac5711638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7092a88ef90c42ffbf1205fac5711638" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 34883 } }
I20260811 18:46:34.357724 7981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:34.350680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36793 (local address 127.30.185.190:34883)
0811 18:46:34.350910 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:34.350918 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:34.351345 (+ 427us) server_negotiation.cc:407] Connection header received
0811 18:46:34.351516 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:34.351524 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:34.351610 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:34.351742 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:34.352251 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:34.353431 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:34.354802 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:34.355451 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:34.355602 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:34.356778 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:34.356785 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:34.356789 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:46:34.356812 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:34.356861 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:46:34.356883 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:34.356984 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:34.357205 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 18:46:34.357314 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 18:46:34.357788 7976 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7092a88ef90c42ffbf1205fac5711638 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:46:34.357959 7976 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7092a88ef90c42ffbf1205fac5711638 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:46:34.358167 7976 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7092a88ef90c42ffbf1205fac5711638 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:46:34.360713 7976 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7092a88ef90c42ffbf1205fac5711638 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7092a88ef90c42ffbf1205fac5711638" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 34883 } }
I20260811 18:46:34.361260 7976 leader_election.cc:302] T 00000000000000000000000000000000 P 7092a88ef90c42ffbf1205fac5711638 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7092a88ef90c42ffbf1205fac5711638; no voters:
I20260811 18:46:34.361814 7976 leader_election.cc:288] T 00000000000000000000000000000000 P 7092a88ef90c42ffbf1205fac5711638 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:46:34.361984 7996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7092a88ef90c42ffbf1205fac5711638 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:46:34.362700 7996 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7092a88ef90c42ffbf1205fac5711638 [term 1 LEADER]: Becoming Leader. State: Replica: 7092a88ef90c42ffbf1205fac5711638, State: Running, Role: LEADER
I20260811 18:46:34.363354 7996 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7092a88ef90c42ffbf1205fac5711638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7092a88ef90c42ffbf1205fac5711638" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 34883 } }
I20260811 18:46:34.364069 7976 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7092a88ef90c42ffbf1205fac5711638 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:46:34.366174 8000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7092a88ef90c42ffbf1205fac5711638 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7092a88ef90c42ffbf1205fac5711638. Latest consensus state: current_term: 1 leader_uuid: "7092a88ef90c42ffbf1205fac5711638" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7092a88ef90c42ffbf1205fac5711638" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 34883 } } }
I20260811 18:46:34.366585 8000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7092a88ef90c42ffbf1205fac5711638 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:34.367550 7999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7092a88ef90c42ffbf1205fac5711638 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7092a88ef90c42ffbf1205fac5711638" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7092a88ef90c42ffbf1205fac5711638" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 34883 } } }
I20260811 18:46:34.368002 7999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7092a88ef90c42ffbf1205fac5711638 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:34.369470 8008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:46:34.375476 8008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:46:34.385200 8008 catalog_manager.cc:1347] Generated new cluster ID: bd52274ddab04d31b3eee61b37c94c4a
I20260811 18:46:34.385351 8008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:46:34.408787 8008 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:46:34.410252 8008 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:46:34.420625 8008 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7092a88ef90c42ffbf1205fac5711638: Generated new TSK 0
I20260811 18:46:34.421622 8008 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:46:34.437587 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 7828
W20260811 18:46:34.438881 7968 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56325
W20260811 18:46:34.439729 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 7828: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:46:34.439893 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 7828
2026-08-11T18:46:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2406 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:46:34 dist-test-slave-z8x9 krb5kdc[8035](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:46:34 dist-test-slave-z8x9 krb5kdc[8035](info): set up 2 sockets
Aug 11 18:46:34 dist-test-slave-z8x9 krb5kdc[8035](info): commencing operation
krb5kdc: starting...
W20260811 18:46:35.876955 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.383s user 0.000s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:46:36 dist-test-slave-z8x9 krb5kdc[8035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:46:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:46:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:46:36.249802 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:33745
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:43981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:33745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:46:36.622534 8061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:46:36.622895 8061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:46:36.623056 8061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:46:36.630599 8061 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:46:36.630779 8061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:46:36.630890 8061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:46:36.631006 8061 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:46:36.631085 8061 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:46:36.642131 8061 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:43981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:33745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:33745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:46:36.643575 8061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:46:36.645040 8061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:46:36.657408 8079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:36.657459 8081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:36.657646 8083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:46:36.658253 8061 server_base.cc:1034] running on GCE node
I20260811 18:46:36.658996 8061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:46:36.660699 8061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:46:36.661978 8061 hybrid_clock.cc:648] HybridClock initialized: now 1786473996661895 us; error 62 us; skew 500 ppm
I20260811 18:46:36.662459 8061 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:46:36 dist-test-slave-z8x9 krb5kdc[8035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473996, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:46:36.667019 8061 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:46:36.668936 8061 webserver.cc:460] Webserver started at http://127.30.185.190:36139/ using document root <none> and password file <none>
I20260811 18:46:36.669749 8061 fs_manager.cc:359] Metadata directory not provided
I20260811 18:46:36.669924 8061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:46:36.670324 8061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:46:36.673604 8061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "77153c3a192a4b09a63d3292d472b4e0"
format_stamp: "Formatted at 2026-08-11 18:46:36 on dist-test-slave-z8x9"
server_key: "5c4467c582b847ce4bb52898f171fd11"
server_key_iv: "abcc40179c8ce4f23dc238d17f94eecc"
server_key_version: "encryptionkey@0"
I20260811 18:46:36.674376 8061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "77153c3a192a4b09a63d3292d472b4e0"
format_stamp: "Formatted at 2026-08-11 18:46:36 on dist-test-slave-z8x9"
server_key: "5c4467c582b847ce4bb52898f171fd11"
server_key_iv: "abcc40179c8ce4f23dc238d17f94eecc"
server_key_version: "encryptionkey@0"
I20260811 18:46:36.680254 8061 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260811 18:46:36.683866 8099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:46:36.685281 8061 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:46:36.685487 8061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "77153c3a192a4b09a63d3292d472b4e0"
format_stamp: "Formatted at 2026-08-11 18:46:36 on dist-test-slave-z8x9"
server_key: "5c4467c582b847ce4bb52898f171fd11"
server_key_iv: "abcc40179c8ce4f23dc238d17f94eecc"
server_key_version: "encryptionkey@0"
I20260811 18:46:36.685698 8061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:46:36.691864 8061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:46:36.696080 8061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:46:36.696395 8061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:46:36.716073 8061 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:33745
I20260811 18:46:36.716099 8214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:33745 every 8 connection(s)
I20260811 18:46:36.717548 8061 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:46:36.721359 8215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:46:36.726146 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 8061
I20260811 18:46:36.726361 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:46:36.726830 31462 external_mini_cluster.cc:1428] Setting key 766e4defa8926de4619f02b2db5bd73b
I20260811 18:46:36.732702 8215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77153c3a192a4b09a63d3292d472b4e0: Bootstrap starting.
I20260811 18:46:36.735086 8219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:36.728698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46751 (local address 127.30.185.190:33745)
0811 18:46:36.729614 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:36.729629 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:36.729669 (+ 40us) server_negotiation.cc:407] Connection header received
0811 18:46:36.731404 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:36.731428 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:36.731759 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:36.732158 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:36.733265 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:36.733275 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:36.733302 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:46:36.733350 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:36.733434 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:46:36.733466 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:36.733694 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:36.733911 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 18:46:36.734117 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":469,"threads_started":1}
I20260811 18:46:36.736374 8215 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77153c3a192a4b09a63d3292d472b4e0: Neither blocks nor log segments found. Creating new log.
I20260811 18:46:36.737813 8215 log.cc:824] T 00000000000000000000000000000000 P 77153c3a192a4b09a63d3292d472b4e0: Log is configured to *not* fsync() on all Append() calls
I20260811 18:46:36.739066 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:46:36.741091 8215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77153c3a192a4b09a63d3292d472b4e0: No bootstrap required, opened a new log
I20260811 18:46:36.747061 8219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:36.744324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52999 (local address 127.30.185.190:33745)
0811 18:46:36.744561 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:36.744570 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:36.745030 (+ 460us) server_negotiation.cc:407] Connection header received
0811 18:46:36.745235 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:36.745239 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:36.745288 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:36.745377 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:36.746360 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:36.746364 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:36.746367 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:46:36.746381 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:36.746421 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:46:36.746436 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:36.746535 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:36.746666 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 18:46:36.746714 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 18:46:36.750859 8215 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77153c3a192a4b09a63d3292d472b4e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77153c3a192a4b09a63d3292d472b4e0" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33745 } }
I20260811 18:46:36.751259 8215 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77153c3a192a4b09a63d3292d472b4e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:46:36.751391 8215 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77153c3a192a4b09a63d3292d472b4e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77153c3a192a4b09a63d3292d472b4e0, State: Initialized, Role: FOLLOWER
I20260811 18:46:36.751964 8215 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77153c3a192a4b09a63d3292d472b4e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77153c3a192a4b09a63d3292d472b4e0" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33745 } }
I20260811 18:46:36.752243 8215 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77153c3a192a4b09a63d3292d472b4e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:46:36.752400 8215 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77153c3a192a4b09a63d3292d472b4e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:46:36.752602 8215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77153c3a192a4b09a63d3292d472b4e0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:46:36.754911 8215 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77153c3a192a4b09a63d3292d472b4e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77153c3a192a4b09a63d3292d472b4e0" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33745 } }
I20260811 18:46:36.755424 8215 leader_election.cc:302] T 00000000000000000000000000000000 P 77153c3a192a4b09a63d3292d472b4e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77153c3a192a4b09a63d3292d472b4e0; no voters:
I20260811 18:46:36.756103 8215 leader_election.cc:288] T 00000000000000000000000000000000 P 77153c3a192a4b09a63d3292d472b4e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:46:36.756242 8233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77153c3a192a4b09a63d3292d472b4e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:46:36.756937 8233 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77153c3a192a4b09a63d3292d472b4e0 [term 1 LEADER]: Becoming Leader. State: Replica: 77153c3a192a4b09a63d3292d472b4e0, State: Running, Role: LEADER
I20260811 18:46:36.757633 8233 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77153c3a192a4b09a63d3292d472b4e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77153c3a192a4b09a63d3292d472b4e0" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33745 } }
I20260811 18:46:36.758455 8215 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77153c3a192a4b09a63d3292d472b4e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:46:36.760499 8234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77153c3a192a4b09a63d3292d472b4e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77153c3a192a4b09a63d3292d472b4e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77153c3a192a4b09a63d3292d472b4e0" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33745 } } }
I20260811 18:46:36.761037 8234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77153c3a192a4b09a63d3292d472b4e0 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:36.760489 8236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77153c3a192a4b09a63d3292d472b4e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77153c3a192a4b09a63d3292d472b4e0. Latest consensus state: current_term: 1 leader_uuid: "77153c3a192a4b09a63d3292d472b4e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77153c3a192a4b09a63d3292d472b4e0" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 33745 } } }
I20260811 18:46:36.761229 8236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77153c3a192a4b09a63d3292d472b4e0 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:36.763500 8242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:46:36.768285 8242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:46:36.777062 8242 catalog_manager.cc:1347] Generated new cluster ID: f8aef15548ac49d8a8a6f8ca6396ee17
I20260811 18:46:36.777244 8242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:46:36.797260 8242 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:46:36.797745 8242 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:46:36.809093 8242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77153c3a192a4b09a63d3292d472b4e0: Generated new TSK 0
I20260811 18:46:36.810174 8242 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:46:36.827564 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 8061
W20260811 18:46:36.828893 8209 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46751
W20260811 18:46:36.829659 31462 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 8061: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 18:46:36.829802 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 8061
2026-08-11T18:46:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2390 ms)
[----------] 18 tests from AuthTokenIssuingTest (44106 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:46:36 dist-test-slave-z8x9 krb5kdc[8281](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:46:36 dist-test-slave-z8x9 krb5kdc[8281](info): set up 2 sockets
Aug 11 18:46:36 dist-test-slave-z8x9 krb5kdc[8281](info): commencing operation
krb5kdc: starting...
W20260811 18:46:38.263221 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.378s user 0.005s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:46:38 dist-test-slave-z8x9 krb5kdc[8281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:46:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:46:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:46:38.641211 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:46669
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:36107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:46669,127.30.185.189:44827,127.30.185.188:37587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:46:38.997083 8300 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:46:38.997498 8300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:46:38.997578 8300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:46:38.997660 8300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:46:39.004933 8300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:46:39.005048 8300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:46:39.005110 8300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:46:39.005163 8300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:46:39.005260 8300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:46:39.015364 8300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:36107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:46669,127.30.185.189:44827,127.30.185.188:37587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:46669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:46:39.016841 8300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:46:39.018252 8300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:46:39.030658 8311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:39.030679 8314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:39.031157 8309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:46:39.032022 8300 server_base.cc:1034] running on GCE node
I20260811 18:46:39.032932 8300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:46:39.034440 8300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:46:39.035701 8300 hybrid_clock.cc:648] HybridClock initialized: now 1786473999035631 us; error 46 us; skew 500 ppm
I20260811 18:46:39.036069 8300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:46:39 dist-test-slave-z8x9 krb5kdc[8281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473999, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:46:39.039548 8300 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:46:39.041015 8300 webserver.cc:460] Webserver started at http://127.30.185.190:40685/ using document root <none> and password file <none>
I20260811 18:46:39.041783 8300 fs_manager.cc:359] Metadata directory not provided
I20260811 18:46:39.041883 8300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:46:39.042160 8300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:46:39.045307 8300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "9909ba5b0fc24daa9357c656c3c75916"
format_stamp: "Formatted at 2026-08-11 18:46:39 on dist-test-slave-z8x9"
server_key: "5c2e96b2f5f5b7c3e1036fff71c7d8c3"
server_key_iv: "22da993771f7c4f25fdfe8d3b209fe35"
server_key_version: "encryptionkey@0"
I20260811 18:46:39.046203 8300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "9909ba5b0fc24daa9357c656c3c75916"
format_stamp: "Formatted at 2026-08-11 18:46:39 on dist-test-slave-z8x9"
server_key: "5c2e96b2f5f5b7c3e1036fff71c7d8c3"
server_key_iv: "22da993771f7c4f25fdfe8d3b209fe35"
server_key_version: "encryptionkey@0"
I20260811 18:46:39.051537 8300 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:46:39.055042 8326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:46:39.056407 8300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:46:39.056555 8300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "9909ba5b0fc24daa9357c656c3c75916"
format_stamp: "Formatted at 2026-08-11 18:46:39 on dist-test-slave-z8x9"
server_key: "5c2e96b2f5f5b7c3e1036fff71c7d8c3"
server_key_iv: "22da993771f7c4f25fdfe8d3b209fe35"
server_key_version: "encryptionkey@0"
I20260811 18:46:39.056746 8300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:46:39.071319 8300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:46:39.075348 8300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:46:39.075575 8300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:46:39.095032 8300 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:46669
I20260811 18:46:39.095069 8429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:46669 every 8 connection(s)
I20260811 18:46:39.096536 8300 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:46:39.100114 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 8300
I20260811 18:46:39.100346 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:46:39.100342 8430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:46:39.101056 31462 external_mini_cluster.cc:1428] Setting key 7604bc98dfdf9de9cb2945d55bedf2e9
I20260811 18:46:39.107055 8430 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46669 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:46:39 dist-test-slave-z8x9 krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473999, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:46:39.130261 8434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.110284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55962 (local address 127.30.185.190:46669)
0811 18:46:39.111171 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:39.111191 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:39.111628 (+ 437us) server_negotiation.cc:407] Connection header received
0811 18:46:39.113723 (+ 2095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:39.113729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:39.113959 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:39.114165 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:39.115500 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:39.116702 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:39.118205 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:39.118757 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:39.118979 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:39.122730 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:39.122739 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:39.122751 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:39.122779 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:39.125725 (+ 2946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:39.126450 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:39.126457 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:39.126460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:39.126603 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:39.127018 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:39.127029 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:39.127032 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:39.127533 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:39.127753 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:39.128286 (+ 533us) server_negotiation.cc:299] Negotiation successful
0811 18:46:39.128689 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":85,"server-negotiator.queue_time_us":616,"thread_start_us":390,"threads_started":1}
I20260811 18:46:39.130288 8435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.110294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46669 (local address 127.0.0.1:55962)
0811 18:46:39.111454 (+ 1160us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.111496 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:46:39.113359 (+ 1863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:46:39.114188 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:46:39.114194 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:46:39.114933 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:46:39.115271 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:39.115288 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:39.116875 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:39.116881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:39.118061 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:39.118073 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:39.119008 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:39.119015 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:39.119269 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:39.119976 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:46:39.119998 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:46:39.122495 (+ 2497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:46:39.125869 (+ 3374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:39.125882 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:39.125894 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:39.126296 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:39.126723 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:39.126728 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:39.126732 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:39.126902 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:39.127671 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:46:39.127684 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:46:39.127909 (+ 225us) client_negotiation.cc:769] Sending connection context
0811 18:46:39.128158 (+ 249us) client_negotiation.cc:240] Negotiation successful
0811 18:46:39.128814 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":952,"mutex_wait_us":186,"thread_start_us":751,"threads_started":1}
I20260811 18:46:39.133837 8430 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.185.189@KRBTEST.COM; defaulting to no policy
I20260811 18:46:39.134969 8435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.134564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:44827 (local address 127.0.0.1:36388)
0811 18:46:39.134734 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.134854 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:44827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 18:46:39.135299 8330 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.185.189:44827: connect: Connection refused (error 111)
I20260811 18:46:39.137019 8435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.136659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:44827 (local address 127.0.0.1:36396)
0811 18:46:39.136813 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.136887 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:44827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 18:46:39.137295 8430 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.189:44827: Network error: Client connection negotiation failed: client connection to 127.30.185.189:44827: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.189.keytab.
Entry for principal kudu/127.30.185.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.189.keytab.
I20260811 18:46:39.181908 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.189:44827
--webserver_interface=127.30.185.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.189.keytab
--principal=kudu/127.30.185.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:36107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:46669,127.30.185.189:44827,127.30.185.188:37587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:46:39.186695 8430 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } attempt: 1
I20260811 18:46:39.188038 8435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.187487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:44827 (local address 127.0.0.1:36400)
0811 18:46:39.187727 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.187868 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:44827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 18:46:39.189289 8435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.188801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:44827 (local address 127.0.0.1:36414)
0811 18:46:39.188982 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.189078 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:44827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 18:46:39.189546 8430 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.189:44827: Network error: Client connection negotiation failed: client connection to 127.30.185.189:44827: connect: Connection refused (error 111)
I20260811 18:46:39.257840 8430 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } attempt: 2
I20260811 18:46:39.259179 8435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.258684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:44827 (local address 127.0.0.1:36422)
0811 18:46:39.258940 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.259037 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:44827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 18:46:39.260219 8435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.259842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:44827 (local address 127.0.0.1:36428)
0811 18:46:39.260001 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.260086 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:44827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 18:46:39.260403 8430 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.189:44827: Network error: Client connection negotiation failed: client connection to 127.30.185.189:44827: connect: Connection refused (error 111)
I20260811 18:46:39.351739 8430 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } attempt: 3
I20260811 18:46:39.352927 8435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.352457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:44827 (local address 127.0.0.1:36434)
0811 18:46:39.352664 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.352793 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:44827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 18:46:39.353979 8435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.353620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:44827 (local address 127.0.0.1:36450)
0811 18:46:39.353763 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.353839 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:44827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 18:46:39.354159 8430 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.189:44827: Network error: Client connection negotiation failed: client connection to 127.30.185.189:44827: connect: Connection refused (error 111)
I20260811 18:46:39.497409 8430 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } attempt: 4
I20260811 18:46:39.498641 8435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.498096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:44827 (local address 127.0.0.1:36464)
0811 18:46:39.498417 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.498506 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:44827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260811 18:46:39.499642 8435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.499287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:44827 (local address 127.0.0.1:36472)
0811 18:46:39.499437 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.499523 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:44827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 18:46:39.499835 8430 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.189:44827: Network error: Client connection negotiation failed: client connection to 127.30.185.189:44827: connect: Connection refused (error 111)
W20260811 18:46:39.538954 8443 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:46:39.539350 8443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:46:39.539479 8443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:46:39.539616 8443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:46:39.547259 8443 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:46:39.547399 8443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:46:39.547509 8443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:46:39.547607 8443 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:46:39.547698 8443 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:46:39.557911 8443 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:36107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:46669,127.30.185.189:44827,127.30.185.188:37587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.189.keytab
--principal=kudu/127.30.185.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.189:44827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.185.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:46:39.559311 8443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:46:39.560714 8443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:46:39.573370 8454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:39.573406 8456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:39.573589 8461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:46:39.574244 8443 server_base.cc:1034] running on GCE node
I20260811 18:46:39.575070 8443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:46:39.576454 8443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:46:39.577698 8443 hybrid_clock.cc:648] HybridClock initialized: now 1786473999577617 us; error 58 us; skew 500 ppm
I20260811 18:46:39.578075 8443 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:46:39 dist-test-slave-z8x9 krb5kdc[8281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473999, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:46:39.581260 8443 init.cc:377] Logged in from keytab as kudu/127.30.185.189@KRBTEST.COM (short username kudu)
I20260811 18:46:39.582715 8443 webserver.cc:460] Webserver started at http://127.30.185.189:45377/ using document root <none> and password file <none>
I20260811 18:46:39.583432 8443 fs_manager.cc:359] Metadata directory not provided
I20260811 18:46:39.583570 8443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:46:39.583896 8443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:46:39.586947 8443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-1/data/instance:
uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f"
format_stamp: "Formatted at 2026-08-11 18:46:39 on dist-test-slave-z8x9"
server_key: "6a855ac60b5990e10a6b0717e8df5f29"
server_key_iv: "b3388a32fdff57d5b73af0011887d73e"
server_key_version: "encryptionkey@0"
I20260811 18:46:39.587769 8443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-1/wal/instance:
uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f"
format_stamp: "Formatted at 2026-08-11 18:46:39 on dist-test-slave-z8x9"
server_key: "6a855ac60b5990e10a6b0717e8df5f29"
server_key_iv: "b3388a32fdff57d5b73af0011887d73e"
server_key_version: "encryptionkey@0"
I20260811 18:46:39.592995 8443 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 18:46:39.596407 8469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:46:39.597882 8443 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:46:39.598081 8443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-1/wal
uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f"
format_stamp: "Formatted at 2026-08-11 18:46:39 on dist-test-slave-z8x9"
server_key: "6a855ac60b5990e10a6b0717e8df5f29"
server_key_iv: "b3388a32fdff57d5b73af0011887d73e"
server_key_version: "encryptionkey@0"
I20260811 18:46:39.598304 8443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:46:39.615949 8443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:46:39.620160 8443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:46:39.620471 8443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:46:39.640081 8443 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.189:44827
I20260811 18:46:39.640127 8569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.189:44827 every 8 connection(s)
I20260811 18:46:39.641639 8443 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-1/data/info.pb
I20260811 18:46:39.645336 8570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:46:39.650071 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 8443
I20260811 18:46:39.650259 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-1/wal/instance
I20260811 18:46:39.650840 31462 external_mini_cluster.cc:1428] Setting key 40af70ec2173bacb20412d3dc2f57503
I20260811 18:46:39.653074 8570 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46669 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:46:39 dist-test-slave-z8x9 krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473999, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.189@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:46:39.677862 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.656672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55964 (local address 127.30.185.190:46669)
0811 18:46:39.657276 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:39.657285 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:39.658488 (+ 1203us) server_negotiation.cc:407] Connection header received
0811 18:46:39.661473 (+ 2985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:39.661477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:39.661552 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:39.661654 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:39.663894 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:39.664602 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:39.667021 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:39.667428 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:39.667529 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:39.671467 (+ 3938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:39.671473 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:39.671476 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:39.671533 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:39.674495 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:39.675343 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:39.675347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:39.675349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:39.675474 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:39.675949 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:39.675953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:39.675955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:39.676248 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:39.676427 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:39.677227 (+ 800us) server_negotiation.cc:299] Negotiation successful
0811 18:46:39.677462 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":256,"threads_started":1}
I20260811 18:46:39.678869 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.657459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46669 (local address 127.0.0.1:55964)
0811 18:46:39.658292 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.658342 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 18:46:39.661075 (+ 2733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:46:39.661946 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:46:39.661960 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:46:39.662800 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:46:39.663634 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:39.663660 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:39.664781 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:39.664788 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:39.666869 (+ 2081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:39.666887 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:39.667691 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:39.667700 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:39.668038 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:39.668843 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:46:39.668879 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:46:39.671215 (+ 2336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:46:39.674677 (+ 3462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:39.674687 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:39.674730 (+ 43us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:39.675186 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:39.675603 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:39.675608 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:39.675612 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:39.675814 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:39.676385 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:46:39.676394 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:46:39.676800 (+ 406us) client_negotiation.cc:769] Sending connection context
0811 18:46:39.677074 (+ 274us) client_negotiation.cc:240] Negotiation successful
0811 18:46:39.677647 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":318,"threads_started":1}
I20260811 18:46:39.680644 8570 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.185.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.188@KRBTEST.COM" created.
Aug 11 18:46:39 dist-test-slave-z8x9 krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473999, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.189@KRBTEST.COM for kudu/127.30.185.189@KRBTEST.COM
I20260811 18:46:39.697364 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.681719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36484 (local address 127.30.185.189:44827)
0811 18:46:39.682431 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:39.682437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:39.682464 (+ 27us) server_negotiation.cc:407] Connection header received
0811 18:46:39.682506 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:39.682512 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:39.682680 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:39.682836 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:39.683607 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:39.684712 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:39.685882 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:39.686421 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:39.686560 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:39.691223 (+ 4663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:39.691235 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:39.691249 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:39.691284 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:39.694272 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:39.695126 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:39.695132 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:39.695135 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:39.695254 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:39.695677 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:39.695680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:39.695682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:39.696019 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:39.696186 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:39.696619 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 18:46:39.696935 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":111,"server-negotiator.queue_time_us":564,"thread_start_us":408,"threads_started":1}
I20260811 18:46:39.697378 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.681487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:44827 (local address 127.0.0.1:36484)
0811 18:46:39.681741 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.681768 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 18:46:39.681881 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:46:39.682828 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:46:39.682832 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:46:39.683301 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:46:39.683427 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:39.683435 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:39.684845 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:39.684852 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:39.685747 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:39.685759 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:39.686569 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:39.686575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:39.686778 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:39.687323 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:46:39.687339 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:46:39.690977 (+ 3638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:46:39.694432 (+ 3455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:39.694440 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:39.694446 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:39.694994 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:39.695363 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:39.695370 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:39.695376 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:39.695567 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:39.696148 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:46:39.696156 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:46:39.696319 (+ 163us) client_negotiation.cc:769] Sending connection context
0811 18:46:39.696479 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 18:46:39.696812 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 18:46:39.700309 8570 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:46:39.701582 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.700962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37587 (local address 127.0.0.1:33060)
0811 18:46:39.701154 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.701381 (+ 227us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 18:46:39.701853 8475 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
I20260811 18:46:39.707653 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.707295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37587 (local address 127.0.0.1:33074)
0811 18:46:39.707459 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.707540 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 18:46:39.707998 8570 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:37587: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.188.keytab.
Entry for principal kudu/127.30.185.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.188.keytab.
I20260811 18:46:39.732899 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.188:37587
--webserver_interface=127.30.185.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.188.keytab
--principal=kudu/127.30.185.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:36107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:46669,127.30.185.189:44827,127.30.185.188:37587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:46:39.757382 8570 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } attempt: 1
I20260811 18:46:39.758730 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.758214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37587 (local address 127.0.0.1:33084)
0811 18:46:39.758422 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.758532 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 18:46:39.759749 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.759378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37587 (local address 127.0.0.1:33086)
0811 18:46:39.759550 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.759641 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 18:46:39.759974 8570 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:37587: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
I20260811 18:46:39.799080 8430 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } attempt: 5
Aug 11 18:46:39 dist-test-slave-z8x9 krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473999, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for kudu/127.30.185.189@KRBTEST.COM
I20260811 18:46:39.813678 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.799972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36492 (local address 127.30.185.189:44827)
0811 18:46:39.800164 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:39.800170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:39.800196 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:46:39.800333 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:39.800339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:39.800422 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:39.800544 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:39.801434 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:39.802584 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:39.803869 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:39.804540 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:39.804675 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:39.807964 (+ 3289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:39.807973 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:39.807977 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:39.808017 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:39.810891 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:39.811598 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:39.811606 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:39.811609 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:39.811749 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:39.812192 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:39.812198 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:39.812201 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:39.812508 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:39.812667 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:39.813075 (+ 408us) server_negotiation.cc:299] Negotiation successful
0811 18:46:39.813334 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 18:46:39.813678 8435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.799870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:44827 (local address 127.0.0.1:36492)
0811 18:46:39.800071 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.800099 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 18:46:39.800205 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:46:39.800591 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:46:39.800594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:46:39.800996 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:46:39.801274 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:39.801285 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:39.802744 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:39.802748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:39.803708 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:39.803722 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:39.804690 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:39.804694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:39.804868 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:39.805595 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:46:39.805614 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:46:39.807735 (+ 2121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:46:39.811042 (+ 3307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:39.811048 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:39.811052 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:39.811473 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:39.811879 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:39.811884 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:39.811890 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:39.812078 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:39.812634 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:46:39.812639 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:46:39.812803 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 18:46:39.812960 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 18:46:39.813315 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 18:46:39.814909 8430 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:46:39.816053 8435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.815595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37587 (local address 127.0.0.1:33102)
0811 18:46:39.815785 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.815882 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 18:46:39.817047 8435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.816633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37587 (local address 127.0.0.1:33104)
0811 18:46:39.816780 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.816900 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 18:46:39.817328 8430 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:37587: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
I20260811 18:46:39.828194 8570 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } attempt: 2
I20260811 18:46:39.829303 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.828797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37587 (local address 127.0.0.1:33110)
0811 18:46:39.828995 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.829086 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 18:46:39.830242 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.829910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37587 (local address 127.0.0.1:33122)
0811 18:46:39.830061 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.830137 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 18:46:39.830489 8570 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:37587: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
I20260811 18:46:39.868690 8430 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } attempt: 1
I20260811 18:46:39.870065 8435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.869479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37587 (local address 127.0.0.1:33138)
0811 18:46:39.869721 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.869842 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 18:46:39.871093 8435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.870728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37587 (local address 127.0.0.1:33150)
0811 18:46:39.870877 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.870962 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 18:46:39.871313 8430 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:37587: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
I20260811 18:46:39.921792 8570 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } attempt: 3
I20260811 18:46:39.923038 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.922528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37587 (local address 127.0.0.1:33164)
0811 18:46:39.922756 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.922868 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 18:46:39.924173 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.923730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37587 (local address 127.0.0.1:33168)
0811 18:46:39.923955 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.924052 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260811 18:46:39.924407 8570 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:37587: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
I20260811 18:46:39.939563 8430 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } attempt: 2
I20260811 18:46:39.941107 8435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.940384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37587 (local address 127.0.0.1:33180)
0811 18:46:39.940687 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.940842 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 18:46:39.942260 8435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:39.941867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37587 (local address 127.0.0.1:33186)
0811 18:46:39.942025 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:39.942114 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 18:46:39.942504 8430 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:37587: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
I20260811 18:46:40.048792 8430 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } attempt: 3
I20260811 18:46:40.050138 8435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:40.049534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37587 (local address 127.0.0.1:33190)
0811 18:46:40.049792 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:40.049914 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 18:46:40.051292 8435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:40.050819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37587 (local address 127.0.0.1:33192)
0811 18:46:40.051046 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:40.051145 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260811 18:46:40.051519 8430 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:37587: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
I20260811 18:46:40.067616 8570 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } attempt: 4
I20260811 18:46:40.068835 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:40.068317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37587 (local address 127.0.0.1:33196)
0811 18:46:40.068578 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:40.068667 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 18:46:40.069866 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:40.069536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37587 (local address 127.0.0.1:33198)
0811 18:46:40.069680 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:40.069757 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 18:46:40.070142 8570 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:37587: Network error: Client connection negotiation failed: client connection to 127.30.185.188:37587: connect: Connection refused (error 111)
W20260811 18:46:40.101354 8581 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:46:40.101814 8581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:46:40.101938 8581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:46:40.102063 8581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:46:40.111392 8581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:46:40.111500 8581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:46:40.111560 8581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:46:40.111613 8581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:46:40.111675 8581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:46:40.121997 8581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:36107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:46669,127.30.185.189:44827,127.30.185.188:37587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.188.keytab
--principal=kudu/127.30.185.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.188:37587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.185.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:46:40.123422 8581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:46:40.124739 8581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:46:40.137140 8592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:40.137285 8591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:40.137272 8594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:46:40.138494 8581 server_base.cc:1034] running on GCE node
I20260811 18:46:40.139349 8581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:46:40.140882 8581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:46:40.142151 8581 hybrid_clock.cc:648] HybridClock initialized: now 1786474000142048 us; error 77 us; skew 500 ppm
I20260811 18:46:40.142518 8581 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:46:40 dist-test-slave-z8x9 krb5kdc[8281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474000, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:46:40.145915 8581 init.cc:377] Logged in from keytab as kudu/127.30.185.188@KRBTEST.COM (short username kudu)
I20260811 18:46:40.147465 8581 webserver.cc:460] Webserver started at http://127.30.185.188:43637/ using document root <none> and password file <none>
I20260811 18:46:40.148293 8581 fs_manager.cc:359] Metadata directory not provided
I20260811 18:46:40.148423 8581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:46:40.148774 8581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:46:40.152024 8581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-2/data/instance:
uuid: "ff17090bc33a4a0aa05419ba89e3d880"
format_stamp: "Formatted at 2026-08-11 18:46:40 on dist-test-slave-z8x9"
server_key: "556ef309c8cf9fec5f22a2121ab1e981"
server_key_iv: "bb25a601554e4d3332968f0773c7d47b"
server_key_version: "encryptionkey@0"
I20260811 18:46:40.152892 8581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-2/wal/instance:
uuid: "ff17090bc33a4a0aa05419ba89e3d880"
format_stamp: "Formatted at 2026-08-11 18:46:40 on dist-test-slave-z8x9"
server_key: "556ef309c8cf9fec5f22a2121ab1e981"
server_key_iv: "bb25a601554e4d3332968f0773c7d47b"
server_key_version: "encryptionkey@0"
I20260811 18:46:40.158536 8581 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 18:46:40.162462 8609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:46:40.164085 8581 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:46:40.164320 8581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-2/wal
uuid: "ff17090bc33a4a0aa05419ba89e3d880"
format_stamp: "Formatted at 2026-08-11 18:46:40 on dist-test-slave-z8x9"
server_key: "556ef309c8cf9fec5f22a2121ab1e981"
server_key_iv: "bb25a601554e4d3332968f0773c7d47b"
server_key_version: "encryptionkey@0"
I20260811 18:46:40.164613 8581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:46:40.199718 8581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:46:40.204465 8581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:46:40.204792 8581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:46:40.225987 8581 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.188:37587
I20260811 18:46:40.226007 8718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.188:37587 every 8 connection(s)
I20260811 18:46:40.227556 8581 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-2/data/info.pb
I20260811 18:46:40.228765 8430 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } attempt: 4
I20260811 18:46:40.230391 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 8581
I20260811 18:46:40.230598 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786473610573616-31462-0/minicluster-data/master-2/wal/instance
I20260811 18:46:40.231210 31462 external_mini_cluster.cc:1428] Setting key 7f44d923e2e5b5c675088838309bc3ab
I20260811 18:46:40.232304 8719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:46:40.245378 8719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46669 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:46:40 dist-test-slave-z8x9 krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473999, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for kudu/127.30.185.188@KRBTEST.COM
Aug 11 18:46:40 dist-test-slave-z8x9 krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.188@KRBTEST.COM
I20260811 18:46:40.259266 8435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:40.229557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37587 (local address 127.0.0.1:33200)
0811 18:46:40.229802 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:40.229834 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 18:46:40.229986 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:46:40.236906 (+ 6920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:46:40.236916 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:46:40.237799 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:46:40.238121 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:40.238140 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.242706 (+ 4566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.242717 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:40.244542 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:40.244563 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.245835 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.245842 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:40.246040 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:40.246819 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:46:40.246838 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:46:40.249839 (+ 3001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:46:40.254951 (+ 5112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:40.254958 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:40.254964 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:40.255539 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:40.256841 (+ 1302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:40.256849 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:40.256862 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:40.257050 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:40.258212 (+ 1162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:46:40.258217 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:46:40.258416 (+ 199us) client_negotiation.cc:769] Sending connection context
0811 18:46:40.258589 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 18:46:40.258895 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 18:46:40.260931 8725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:40.230680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33200 (local address 127.30.185.188:37587)
0811 18:46:40.231699 (+ 1019us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:40.231717 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:40.231778 (+ 61us) server_negotiation.cc:407] Connection header received
0811 18:46:40.235410 (+ 3632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:40.235454 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:40.236222 (+ 768us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:40.238532 (+ 2310us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:40.239367 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.242433 (+ 3066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.244740 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.245644 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.245882 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:40.251015 (+ 5133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:40.251027 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:40.251046 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:40.251096 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:40.254769 (+ 3673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:40.255698 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:40.255718 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:40.255728 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:40.256001 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:40.257253 (+ 1252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:40.257259 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:40.257264 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:40.258059 (+ 795us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:40.258247 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:40.258884 (+ 637us) server_negotiation.cc:299] Negotiation successful
0811 18:46:40.259406 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":411,"threads_started":1}
Aug 11 18:46:40 dist-test-slave-z8x9 krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474000, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.188@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:46:40.265261 8730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:40.232629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33204 (local address 127.30.185.188:37587)
0811 18:46:40.243365 (+ 10736us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:40.243372 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:40.244121 (+ 749us) server_negotiation.cc:407] Connection header received
0811 18:46:40.244310 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:40.244317 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:40.244404 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:40.244530 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:40.246561 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.249450 (+ 2889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.251764 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.252533 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.252676 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:40.258175 (+ 5499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:40.258182 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:40.258187 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:40.258220 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:40.262152 (+ 3932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:40.263069 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:40.263075 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:40.263078 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:40.263224 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:40.263654 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:40.263662 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:40.263668 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:40.264028 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:40.264322 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:40.264629 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 18:46:40.264847 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10561,"thread_start_us":10740,"threads_started":1}
I20260811 18:46:40.269421 8430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916: Bootstrap starting.
I20260811 18:46:40.270234 8733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:40.248491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:46669 (local address 127.0.0.1:55980)
0811 18:46:40.250526 (+ 2035us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:40.250568 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:46:40.250707 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:46:40.251633 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:46:40.251641 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:46:40.252925 (+ 1284us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:46:40.253386 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:40.253398 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.255032 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.255039 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:40.256542 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:40.256558 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.257959 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.257969 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:40.258214 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:40.259312 (+ 1098us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:46:40.259343 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:46:40.262672 (+ 3329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:46:40.267025 (+ 4353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:40.267036 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:40.267042 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:40.267582 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:40.268049 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:40.268053 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:40.268056 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:40.268219 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:40.269009 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:46:40.269018 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:46:40.269241 (+ 223us) client_negotiation.cc:769] Sending connection context
0811 18:46:40.269479 (+ 238us) client_negotiation.cc:240] Negotiation successful
0811 18:46:40.269838 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1800,"thread_start_us":1661,"threads_started":1}
I20260811 18:46:40.271492 8731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:40.248602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55980 (local address 127.30.185.190:46669)
0811 18:46:40.251287 (+ 2685us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:40.251293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:40.251310 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:46:40.251358 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:40.251363 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:40.251432 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:40.251546 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:40.253552 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.254849 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.256722 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.257537 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.257672 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:40.262887 (+ 5215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:40.262894 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:40.262899 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:40.262939 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:40.266843 (+ 3904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:40.267729 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:40.267735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:40.267738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:40.267907 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:40.268354 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:40.268359 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:40.268361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:40.268740 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:40.268929 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:40.270769 (+ 1840us) server_negotiation.cc:299] Negotiation successful
0811 18:46:40.271004 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2528,"thread_start_us":2377,"threads_started":1}
I20260811 18:46:40.272733 8719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:46:40.274379 8430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916: Neither blocks nor log segments found. Creating new log.
I20260811 18:46:40.276106 8430 log.cc:824] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:46:40 dist-test-slave-z8x9 krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474000, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.188@KRBTEST.COM for kudu/127.30.185.189@KRBTEST.COM
I20260811 18:46:40.284329 8430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916: No bootstrap required, opened a new log
Aug 11 18:46:40 dist-test-slave-z8x9 krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:46:40.292552 8731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:40.267066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55996 (local address 127.30.185.190:46669)
0811 18:46:40.271738 (+ 4672us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:40.271744 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:40.271759 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:46:40.271824 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:40.271828 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:40.271925 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:40.272024 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:40.273998 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.275597 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.278497 (+ 2900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.279581 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.279761 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:40.285874 (+ 6113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:40.285880 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:40.285883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:40.285917 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:40.289461 (+ 3544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:40.290189 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:40.290196 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:40.290201 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:40.290347 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:40.290818 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:40.290822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:40.290824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:40.291164 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:40.291494 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:40.291840 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 18:46:40.292125 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4556,"thread_start_us":12682,"threads_started":1}
I20260811 18:46:40.292657 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:40.274055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36496 (local address 127.30.185.189:44827)
0811 18:46:40.274266 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:40.274272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:40.274521 (+ 249us) server_negotiation.cc:407] Connection header received
0811 18:46:40.274773 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:40.274781 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:40.274885 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:40.275014 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:40.276210 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.277462 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.279755 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.280563 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.280719 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:40.285634 (+ 4915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:40.285641 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:40.285646 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:40.285691 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:40.289439 (+ 3748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:40.290313 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:40.290319 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:40.290323 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:40.290490 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:40.290958 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:40.290965 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:40.290968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:40.291321 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:40.291452 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:40.292024 (+ 572us) server_negotiation.cc:299] Negotiation successful
0811 18:46:40.292256 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 18:46:40.293629 8733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:40.273653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:44827 (local address 127.0.0.1:36496)
0811 18:46:40.274428 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:40.274448 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 18:46:40.274617 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:46:40.275224 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:46:40.275229 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:46:40.275829 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:46:40.276014 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:40.276030 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.277715 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.277723 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:40.279571 (+ 1848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:40.279590 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.281058 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.281067 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:40.281384 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:40.282043 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:46:40.282060 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:46:40.285397 (+ 3337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:46:40.289614 (+ 4217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:40.289619 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:40.289624 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:40.290162 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:40.290646 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:40.290650 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:40.290654 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:40.290828 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:40.291552 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:46:40.291557 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:46:40.291727 (+ 170us) client_negotiation.cc:769] Sending connection context
0811 18:46:40.292910 (+ 1183us) client_negotiation.cc:240] Negotiation successful
0811 18:46:40.293249 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643}
I20260811 18:46:40.295277 8719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:46:40.296366 8430 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9909ba5b0fc24daa9357c656c3c75916" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46669 } } peers { permanent_uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } } peers { permanent_uuid: "ff17090bc33a4a0aa05419ba89e3d880" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } }
I20260811 18:46:40.296815 8430 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9909ba5b0fc24daa9357c656c3c75916, State: Initialized, Role: FOLLOWER
I20260811 18:46:40.297449 8430 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9909ba5b0fc24daa9357c656c3c75916" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46669 } } peers { permanent_uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } } peers { permanent_uuid: "ff17090bc33a4a0aa05419ba89e3d880" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } }
I20260811 18:46:40.298497 8737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9909ba5b0fc24daa9357c656c3c75916" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46669 } } peers { permanent_uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } } peers { permanent_uuid: "ff17090bc33a4a0aa05419ba89e3d880" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } } }
I20260811 18:46:40.299260 8737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:46:40.299957 8430 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916 [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 18:46:40 dist-test-slave-z8x9 krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474000, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.188@KRBTEST.COM for kudu/127.30.185.188@KRBTEST.COM
Aug 11 18:46:40 dist-test-slave-z8x9 krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.189@KRBTEST.COM
I20260811 18:46:40.313676 8733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:40.295937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37587 (local address 127.0.0.1:33218)
0811 18:46:40.296086 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:40.296102 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:46:40.296184 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:46:40.296598 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:46:40.296601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:46:40.297221 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:46:40.297410 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:40.297419 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.298925 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.298933 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:40.300130 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:40.300145 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.300930 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.300936 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:40.301229 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:40.302119 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:46:40.302146 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:46:40.304870 (+ 2724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:46:40.310306 (+ 5436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:40.310312 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:40.310317 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:40.310884 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:40.311369 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:40.311377 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:40.311384 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:40.311608 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:40.312516 (+ 908us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:46:40.312521 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:46:40.312682 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 18:46:40.312875 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 18:46:40.313280 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 18:46:40.315075 8730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:40.296137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33218 (local address 127.30.185.188:37587)
0811 18:46:40.296282 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:40.296288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:40.296307 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:46:40.296386 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:40.296394 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:40.296465 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:40.296570 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:40.297559 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.298752 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.300297 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.300758 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.300909 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:40.305069 (+ 4160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:40.305075 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:40.305079 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:40.305109 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:40.310138 (+ 5029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:40.311055 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:40.311059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:40.311063 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:40.311229 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:40.311801 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:40.311809 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:40.311816 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:40.312362 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:40.314290 (+ 1928us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:40.314453 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 18:46:40.314713 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 18:46:40.316642 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:40.294881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36500 (local address 127.30.185.189:44827)
0811 18:46:40.295060 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:40.295067 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:40.295091 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:46:40.295146 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:40.295152 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:40.295236 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:40.295347 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:40.298157 (+ 2810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.299192 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.301020 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.301885 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.302010 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:40.308979 (+ 6969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:40.308987 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:40.308993 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:40.309046 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:40.312617 (+ 3571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:40.313566 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:40.313572 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:40.313575 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:40.313774 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:40.314453 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:40.314459 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:40.314462 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:40.314828 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:40.315098 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:40.316061 (+ 963us) server_negotiation.cc:299] Negotiation successful
0811 18:46:40.316306 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 18:46:40.322604 8719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff17090bc33a4a0aa05419ba89e3d880: Bootstrap starting.
W20260811 18:46:40.324142 8753 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:46:40.324379 8753 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:46:40.326465 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:46:40.327144 8719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff17090bc33a4a0aa05419ba89e3d880: Neither blocks nor log segments found. Creating new log.
I20260811 18:46:40.328660 8719 log.cc:824] T 00000000000000000000000000000000 P ff17090bc33a4a0aa05419ba89e3d880: Log is configured to *not* fsync() on all Append() calls
I20260811 18:46:40.314256 8300 master_runner.cc:186] Error getting master registration for 127.30.185.189:44827: OK, instance_id { permanent_uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f" instance_seqno: 1786473999634169 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:46:40.329183 8300 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 18:46:40.329511 8334 tablet.cc:2369] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:46:40.330722 8375 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "9909ba5b0fc24daa9357c656c3c75916"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55996
I20260811 18:46:40.331311 8375 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 18:46:40.331574 8375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:46:40.334141 8719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff17090bc33a4a0aa05419ba89e3d880: No bootstrap required, opened a new log
I20260811 18:46:40.334548 8375 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9909ba5b0fc24daa9357c656c3c75916" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46669 } } peers { permanent_uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } } peers { permanent_uuid: "ff17090bc33a4a0aa05419ba89e3d880" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } }
I20260811 18:46:40.337414 8375 leader_election.cc:288] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916 [CANDIDATE]: Term 1 election: Requested vote from peers 01c2dbb642a944ecb3d8e9fd4c80d57f (127.30.185.189:44827), ff17090bc33a4a0aa05419ba89e3d880 (127.30.185.188:37587)
I20260811 18:46:40.345811 8719 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff17090bc33a4a0aa05419ba89e3d880 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9909ba5b0fc24daa9357c656c3c75916" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46669 } } peers { permanent_uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } } peers { permanent_uuid: "ff17090bc33a4a0aa05419ba89e3d880" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } }
I20260811 18:46:40.346407 8719 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff17090bc33a4a0aa05419ba89e3d880 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff17090bc33a4a0aa05419ba89e3d880, State: Initialized, Role: FOLLOWER
I20260811 18:46:40.346985 8719 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff17090bc33a4a0aa05419ba89e3d880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9909ba5b0fc24daa9357c656c3c75916" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46669 } } peers { permanent_uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } } peers { permanent_uuid: "ff17090bc33a4a0aa05419ba89e3d880" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } }
I20260811 18:46:40.357887 8773 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff17090bc33a4a0aa05419ba89e3d880 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9909ba5b0fc24daa9357c656c3c75916" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46669 } } peers { permanent_uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } } peers { permanent_uuid: "ff17090bc33a4a0aa05419ba89e3d880" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } } }
I20260811 18:46:40.359647 8773 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff17090bc33a4a0aa05419ba89e3d880 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:46:40.359475 8719 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff17090bc33a4a0aa05419ba89e3d880 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:46:40.360329 8661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9909ba5b0fc24daa9357c656c3c75916" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ff17090bc33a4a0aa05419ba89e3d880"
I20260811 18:46:40.361121 8661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff17090bc33a4a0aa05419ba89e3d880 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:46:40.367921 8661 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ff17090bc33a4a0aa05419ba89e3d880 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9909ba5b0fc24daa9357c656c3c75916 in term 1.
I20260811 18:46:40.369351 8570 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } attempt: 5
I20260811 18:46:40.369350 8331 leader_election.cc:302] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9909ba5b0fc24daa9357c656c3c75916, ff17090bc33a4a0aa05419ba89e3d880; no voters:
I20260811 18:46:40.370036 8737 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916 [term 1 FOLLOWER]: Leader election won for term 1
Aug 11 18:46:40 dist-test-slave-z8x9 krb5kdc[8281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786473999, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.189@KRBTEST.COM for kudu/127.30.185.188@KRBTEST.COM
I20260811 18:46:40.394578 8574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:40.370089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:37587 (local address 127.0.0.1:33228)
0811 18:46:40.370541 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:40.370564 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 18:46:40.370715 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:46:40.371962 (+ 1247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:46:40.371971 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:46:40.372692 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:46:40.372986 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:40.373005 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.375084 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.375091 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:40.377229 (+ 2138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:40.377249 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.378941 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.378949 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:40.379281 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:40.380246 (+ 965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:46:40.380272 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:46:40.384054 (+ 3782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:46:40.389455 (+ 5401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:40.389465 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:40.389471 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:40.390166 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:40.391599 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:40.391606 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:40.391612 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:40.391901 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:40.393145 (+ 1244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:46:40.393154 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:46:40.393383 (+ 229us) client_negotiation.cc:769] Sending connection context
0811 18:46:40.393575 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 18:46:40.393966 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287}
I20260811 18:46:40.396360 8730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:40.370981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33228 (local address 127.30.185.188:37587)
0811 18:46:40.371225 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:40.371235 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:40.371257 (+ 22us) server_negotiation.cc:407] Connection header received
0811 18:46:40.371331 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:40.371339 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:40.371442 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:40.371562 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:40.373266 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.374853 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.377434 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.378445 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.378635 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:40.384473 (+ 5838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:40.384484 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:40.384493 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:40.384539 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:40.389146 (+ 4607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:40.390364 (+ 1218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:40.390371 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:40.390375 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:40.390603 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:40.392064 (+ 1461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:40.392070 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:40.392073 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:40.392548 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:40.392748 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:40.395408 (+ 2660us) server_negotiation.cc:299] Negotiation successful
0811 18:46:40.395766 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 18:46:40.404826 8570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01c2dbb642a944ecb3d8e9fd4c80d57f: Bootstrap starting.
I20260811 18:46:40.410873 8570 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01c2dbb642a944ecb3d8e9fd4c80d57f: Neither blocks nor log segments found. Creating new log.
I20260811 18:46:40.411610 8737 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916 [term 1 LEADER]: Becoming Leader. State: Replica: 9909ba5b0fc24daa9357c656c3c75916, State: Running, Role: LEADER
I20260811 18:46:40.413151 8737 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9909ba5b0fc24daa9357c656c3c75916" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46669 } } peers { permanent_uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } } peers { permanent_uuid: "ff17090bc33a4a0aa05419ba89e3d880" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } }
I20260811 18:46:40.414527 8570 log.cc:824] T 00000000000000000000000000000000 P 01c2dbb642a944ecb3d8e9fd4c80d57f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:46:40.430418 8776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9909ba5b0fc24daa9357c656c3c75916. Latest consensus state: current_term: 1 leader_uuid: "9909ba5b0fc24daa9357c656c3c75916" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9909ba5b0fc24daa9357c656c3c75916" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46669 } } peers { permanent_uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } } peers { permanent_uuid: "ff17090bc33a4a0aa05419ba89e3d880" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } } }
I20260811 18:46:40.431030 8776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:40.432258 8570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01c2dbb642a944ecb3d8e9fd4c80d57f: No bootstrap required, opened a new log
I20260811 18:46:40.435500 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:40.343114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36514 (local address 127.30.185.189:44827)
0811 18:46:40.343343 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:40.343352 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:40.344245 (+ 893us) server_negotiation.cc:407] Connection header received
0811 18:46:40.345316 (+ 1071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:40.345327 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:40.345451 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:40.345558 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:40.358291 (+ 12733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.360005 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.392018 (+ 32013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.392806 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.392911 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:40.423217 (+ 30306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:40.423226 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:40.423231 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:40.423282 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:40.429125 (+ 5843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:40.431550 (+ 2425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:40.431561 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:40.431565 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:40.431823 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:40.432886 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:40.432895 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:40.432901 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:40.433469 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:40.433826 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:40.434690 (+ 864us) server_negotiation.cc:299] Negotiation successful
0811 18:46:40.434991 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 18:46:40.441843 8786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:46:40.444211 8786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:46:40.452548 8570 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01c2dbb642a944ecb3d8e9fd4c80d57f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9909ba5b0fc24daa9357c656c3c75916" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46669 } } peers { permanent_uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } } peers { permanent_uuid: "ff17090bc33a4a0aa05419ba89e3d880" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } }
I20260811 18:46:40.453024 8570 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01c2dbb642a944ecb3d8e9fd4c80d57f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01c2dbb642a944ecb3d8e9fd4c80d57f, State: Initialized, Role: FOLLOWER
I20260811 18:46:40.453653 8570 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01c2dbb642a944ecb3d8e9fd4c80d57f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9909ba5b0fc24daa9357c656c3c75916" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46669 } } peers { permanent_uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } } peers { permanent_uuid: "ff17090bc33a4a0aa05419ba89e3d880" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } }
W20260811 18:46:40.457732 8621 tablet.cc:2369] T 00000000000000000000000000000000 P ff17090bc33a4a0aa05419ba89e3d880: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:46:40.457899 8788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01c2dbb642a944ecb3d8e9fd4c80d57f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9909ba5b0fc24daa9357c656c3c75916" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46669 } } peers { permanent_uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } } peers { permanent_uuid: "ff17090bc33a4a0aa05419ba89e3d880" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } } }
I20260811 18:46:40.458361 8788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01c2dbb642a944ecb3d8e9fd4c80d57f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:46:40.462224 8661 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ff17090bc33a4a0aa05419ba89e3d880 [term 1 FOLLOWER]: Refusing update from remote peer 9909ba5b0fc24daa9357c656c3c75916: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:46:40.463418 8776 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff17090bc33a4a0aa05419ba89e3d880" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:46:40.472447 8731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:40.343061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56002 (local address 127.30.185.190:46669)
0811 18:46:40.343260 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:40.343267 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:40.385111 (+ 41844us) server_negotiation.cc:407] Connection header received
0811 18:46:40.386826 (+ 1715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:40.386836 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:40.386947 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:40.387047 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:40.403370 (+ 16323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.404673 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.423955 (+ 19282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.425235 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.425368 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:40.449159 (+ 23791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:40.449188 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:40.449195 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:40.449262 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:40.454775 (+ 5513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:40.464676 (+ 9901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:40.464688 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:40.464695 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:40.464969 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:40.466677 (+ 1708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:40.466687 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:40.466693 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:40.467326 (+ 633us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:40.467689 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:40.471400 (+ 3711us) server_negotiation.cc:299] Negotiation successful
0811 18:46:40.471826 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 18:46:40.473840 8528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9909ba5b0fc24daa9357c656c3c75916" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f"
I20260811 18:46:40.474365 8525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01c2dbb642a944ecb3d8e9fd4c80d57f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:46:40.476963 8725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:40.391438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33232 (local address 127.30.185.188:37587)
0811 18:46:40.391646 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:40.391652 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:40.416917 (+ 25265us) server_negotiation.cc:407] Connection header received
0811 18:46:40.417559 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:40.417569 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:40.417677 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:40.417768 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:40.420257 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.421599 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.438687 (+ 17088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:40.439860 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:40.440009 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:40.456144 (+ 16135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:40.456153 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:40.456159 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:40.456212 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:40.463250 (+ 7038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:40.465745 (+ 2495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:40.465756 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:40.465762 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:40.465996 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:40.468376 (+ 2380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:40.468387 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:40.468394 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:40.469288 (+ 894us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:40.469633 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:40.475939 (+ 6306us) server_negotiation.cc:299] Negotiation successful
0811 18:46:40.476367 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 18:46:40.478593 8525 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 01c2dbb642a944ecb3d8e9fd4c80d57f [term 1 FOLLOWER]: Refusing update from remote peer 9909ba5b0fc24daa9357c656c3c75916: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:46:40.482136 8528 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 01c2dbb642a944ecb3d8e9fd4c80d57f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9909ba5b0fc24daa9357c656c3c75916 in term 1.
I20260811 18:46:40.484935 8776 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:46:40.490227 8570 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01c2dbb642a944ecb3d8e9fd4c80d57f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:46:40.506204 8788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01c2dbb642a944ecb3d8e9fd4c80d57f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9909ba5b0fc24daa9357c656c3c75916. Latest consensus state: current_term: 1 leader_uuid: "9909ba5b0fc24daa9357c656c3c75916" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9909ba5b0fc24daa9357c656c3c75916" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46669 } } peers { permanent_uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } } peers { permanent_uuid: "ff17090bc33a4a0aa05419ba89e3d880" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } } }
I20260811 18:46:40.506738 8788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01c2dbb642a944ecb3d8e9fd4c80d57f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:46:40.510093 8787 mvcc.cc:205] Tried to move back new op lower bound from 7317397505878446080 to 7317397505718874112. Current Snapshot: MvccSnapshot[applied={T|T < 7317397505878446080}]
I20260811 18:46:40.513140 8773 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff17090bc33a4a0aa05419ba89e3d880 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9909ba5b0fc24daa9357c656c3c75916. Latest consensus state: current_term: 1 leader_uuid: "9909ba5b0fc24daa9357c656c3c75916" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9909ba5b0fc24daa9357c656c3c75916" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46669 } } peers { permanent_uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } } peers { permanent_uuid: "ff17090bc33a4a0aa05419ba89e3d880" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } } }
I20260811 18:46:40.513645 8773 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff17090bc33a4a0aa05419ba89e3d880 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:46:40.516992 8792 mvcc.cc:205] Tried to move back new op lower bound from 7317397505878446080 to 7317397505718874112. Current Snapshot: MvccSnapshot[applied={T|T < 7317397505878446080}]
I20260811 18:46:40.518451 8788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01c2dbb642a944ecb3d8e9fd4c80d57f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9909ba5b0fc24daa9357c656c3c75916" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9909ba5b0fc24daa9357c656c3c75916" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46669 } } peers { permanent_uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } } peers { permanent_uuid: "ff17090bc33a4a0aa05419ba89e3d880" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } } }
I20260811 18:46:40.518791 8788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01c2dbb642a944ecb3d8e9fd4c80d57f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:46:40.525936 8776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9909ba5b0fc24daa9357c656c3c75916" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9909ba5b0fc24daa9357c656c3c75916" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46669 } } peers { permanent_uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } } peers { permanent_uuid: "ff17090bc33a4a0aa05419ba89e3d880" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } } }
I20260811 18:46:40.526304 8776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:40.526644 8776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9909ba5b0fc24daa9357c656c3c75916" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9909ba5b0fc24daa9357c656c3c75916" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46669 } } peers { permanent_uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } } peers { permanent_uuid: "ff17090bc33a4a0aa05419ba89e3d880" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } } }
I20260811 18:46:40.526899 8776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:40.527601 8773 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff17090bc33a4a0aa05419ba89e3d880 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9909ba5b0fc24daa9357c656c3c75916" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9909ba5b0fc24daa9357c656c3c75916" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 46669 } } peers { permanent_uuid: "01c2dbb642a944ecb3d8e9fd4c80d57f" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 44827 } } peers { permanent_uuid: "ff17090bc33a4a0aa05419ba89e3d880" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 37587 } } }
I20260811 18:46:40.527870 8773 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff17090bc33a4a0aa05419ba89e3d880 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:46:40.529965 8786 catalog_manager.cc:1347] Generated new cluster ID: 90b08a91e00146f3b85979e3d571f9f2
I20260811 18:46:40.530090 8786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:46:40.550982 8830 catalog_manager.cc:1259] Loaded cluster ID: 90b08a91e00146f3b85979e3d571f9f2
I20260811 18:46:40.551093 8830 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 01c2dbb642a944ecb3d8e9fd4c80d57f: loading cluster ID for follower catalog manager: success
W20260811 18:46:40.552397 8830 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 01c2dbb642a944ecb3d8e9fd4c80d57f: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260811 18:46:40.552537 8830 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:46:40.543069 8443 master_runner.cc:186] Error getting master registration for 127.30.185.188:37587: OK, instance_id { permanent_uuid: "ff17090bc33a4a0aa05419ba89e3d880" instance_seqno: 1786474000219884 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:46:40.558698 8443 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:46:40.579794 8786 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:46:40.582209 8786 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:46:40.607460 8786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9909ba5b0fc24daa9357c656c3c75916: Generated new TSK 0
I20260811 18:46:40.608731 8786 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:46:40.617941 8837 catalog_manager.cc:1259] Loaded cluster ID: 90b08a91e00146f3b85979e3d571f9f2
I20260811 18:46:40.618060 8837 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ff17090bc33a4a0aa05419ba89e3d880: loading cluster ID for follower catalog manager: success
I20260811 18:46:40.656261 8837 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ff17090bc33a4a0aa05419ba89e3d880: acquiring CA information for follower catalog manager: success
I20260811 18:46:40.659281 8837 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ff17090bc33a4a0aa05419ba89e3d880: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 18:46:40.746423 31462 mini_kdc.cc:348] Time spent kdestroy: real 0.115s user 0.003s sys 0.029s
W20260811 18:46:40.801987 8731 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:40.780749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56018 (local address 127.30.185.190:46669)
0811 18:46:40.780980 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:40.780988 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:40.799256 (+ 18268us) server_negotiation.cc:407] Connection header received
0811 18:46:40.799899 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:40.799907 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:40.800003 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:40.800087 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:40.801663 (+ 1576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56018: BlockingRecv error: recv got EOF from 127.0.0.1:56018 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260811 18:46:40.805058 8577 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:40.783164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36524 (local address 127.30.185.189:44827)
0811 18:46:40.783423 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:40.783430 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:40.803212 (+ 19782us) server_negotiation.cc:407] Connection header received
0811 18:46:40.803506 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:40.803516 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:40.803619 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:40.803701 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:40.804761 (+ 1060us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36524: BlockingRecv error: recv got EOF from 127.0.0.1:36524 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 18:46:40.809682 8725 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:40.806574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33246 (local address 127.30.185.188:37587)
0811 18:46:40.806825 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:40.806833 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:40.807851 (+ 1018us) server_negotiation.cc:407] Connection header received
0811 18:46:40.808143 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:40.808151 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:40.808245 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:40.808327 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:40.809395 (+ 1068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33246: BlockingRecv error: recv got EOF from 127.0.0.1:33246 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
W20260811 18:46:40.856331 8577 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:40.842457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36526 (local address 127.30.185.189:44827)
0811 18:46:40.842673 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:40.842680 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:40.853883 (+ 11203us) server_negotiation.cc:407] Connection header received
0811 18:46:40.854328 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:40.854335 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:40.854503 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:40.854613 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:40.856009 (+ 1396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36526: BlockingRecv error: recv got EOF from 127.0.0.1:36526 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 18:46:40.906191 8577 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:40.889968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36534 (local address 127.30.185.189:44827)
0811 18:46:40.890159 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:40.890165 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:40.903956 (+ 13791us) server_negotiation.cc:407] Connection header received
0811 18:46:40.904510 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:40.904524 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:40.904643 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:40.904736 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:40.905903 (+ 1167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36534: BlockingRecv error: recv got EOF from 127.0.0.1:36534 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 18:46:40.951857 8577 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:40.943615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36538 (local address 127.30.185.189:44827)
0811 18:46:40.943823 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:40.943831 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:40.949847 (+ 6016us) server_negotiation.cc:407] Connection header received
0811 18:46:40.950124 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:40.950143 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:40.950268 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:40.950354 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:40.951573 (+ 1219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36538: BlockingRecv error: recv got EOF from 127.0.0.1:36538 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 18:46:40.963168 8577 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:40.960374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36542 (local address 127.30.185.189:44827)
0811 18:46:40.960628 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:40.960635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:40.961258 (+ 623us) server_negotiation.cc:407] Connection header received
0811 18:46:40.961607 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:40.961614 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:40.961720 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:40.961834 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:40.962894 (+ 1060us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36542: BlockingRecv error: recv got EOF from 127.0.0.1:36542 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
W20260811 18:46:40.977914 8577 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:40.975429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36556 (local address 127.30.185.189:44827)
0811 18:46:40.975636 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:40.975643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:40.976241 (+ 598us) server_negotiation.cc:407] Connection header received
0811 18:46:40.976468 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:40.976474 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:40.976574 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:40.976692 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:40.977647 (+ 955us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36556: BlockingRecv error: recv got EOF from 127.0.0.1:36556 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 18:46:41.000873 8577 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:40.998234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36562 (local address 127.30.185.189:44827)
0811 18:46:40.998464 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:40.998471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:40.999071 (+ 600us) server_negotiation.cc:407] Connection header received
0811 18:46:40.999334 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:40.999340 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:40.999469 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:40.999581 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:41.000587 (+ 1006us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36562: BlockingRecv error: recv got EOF from 127.0.0.1:36562 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260811 18:46:41.041810 8577 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:41.039253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36568 (local address 127.30.185.189:44827)
0811 18:46:41.039466 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:41.039472 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:41.040050 (+ 578us) server_negotiation.cc:407] Connection header received
0811 18:46:41.040278 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:41.040286 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:41.040400 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:41.040528 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:41.041499 (+ 971us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36568: BlockingRecv error: recv got EOF from 127.0.0.1:36568 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 18:46:41.112246 8577 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:41.110007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36578 (local address 127.30.185.189:44827)
0811 18:46:41.110206 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:41.110214 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:41.110577 (+ 363us) server_negotiation.cc:407] Connection header received
0811 18:46:41.110883 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:41.110908 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:41.111002 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:41.111117 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:41.112000 (+ 883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36578: BlockingRecv error: recv got EOF from 127.0.0.1:36578 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 18:46:41.248739 8577 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:41.246704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36582 (local address 127.30.185.189:44827)
0811 18:46:41.246977 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:41.246984 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:41.247201 (+ 217us) server_negotiation.cc:407] Connection header received
0811 18:46:41.247369 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:41.247374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:41.247450 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:41.247542 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:41.248470 (+ 928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36582: BlockingRecv error: recv got EOF from 127.0.0.1:36582 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
W20260811 18:46:41.512749 8577 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:41.510815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36586 (local address 127.30.185.189:44827)
0811 18:46:41.511034 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:41.511039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:41.511385 (+ 346us) server_negotiation.cc:407] Connection header received
0811 18:46:41.511557 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:41.511562 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:41.511637 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:41.511739 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:41.512568 (+ 829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36586: BlockingRecv error: recv got EOF from 127.0.0.1:36586 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 18:46:41.555969 8830 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 01c2dbb642a944ecb3d8e9fd4c80d57f: acquiring CA information for follower catalog manager: success
I20260811 18:46:41.557976 8830 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 01c2dbb642a944ecb3d8e9fd4c80d57f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:46:42.036648 8958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:42.030746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36596 (local address 127.30.185.189:44827)
0811 18:46:42.031311 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:42.031319 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:42.031405 (+ 86us) server_negotiation.cc:407] Connection header received
0811 18:46:42.031625 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:42.031629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:42.031786 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:42.031923 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:46:42.032425 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:42.033155 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:42.034486 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:42.034906 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:42.034999 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:42.035505 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:46:42.036033 (+ 528us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:46:42.036161 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:42.036333 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 18:46:42.036404 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":288,"threads_started":1}
I20260811 18:46:42.053362 8971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:42.047604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33250 (local address 127.30.185.188:37587)
0811 18:46:42.048117 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:42.048122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:42.048155 (+ 33us) server_negotiation.cc:407] Connection header received
0811 18:46:42.048406 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:42.048410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:42.048565 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:42.048722 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:46:42.049164 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:42.049946 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:42.051269 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:42.051663 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:42.051752 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:42.052199 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:46:42.052694 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:46:42.052805 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:42.052918 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 18:46:42.052983 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":263,"threads_started":1}
I20260811 18:46:42.067974 8958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:42.060341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36608 (local address 127.30.185.189:44827)
0811 18:46:42.060573 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:42.060579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:42.062058 (+ 1479us) server_negotiation.cc:407] Connection header received
0811 18:46:42.062287 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:42.062296 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:42.062398 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:42.062490 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:46:42.063266 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:42.064297 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:42.066237 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:42.066639 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:42.066783 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:42.067200 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:46:42.067300 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:46:42.067412 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:42.067576 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 18:46:42.067664 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:46:42.069569 8971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:42.061357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33260 (local address 127.30.185.188:37587)
0811 18:46:42.061575 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:42.061584 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:42.063037 (+ 1453us) server_negotiation.cc:407] Connection header received
0811 18:46:42.063307 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:42.063312 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:42.063401 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:42.063474 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:46:42.064134 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:42.065171 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:42.067529 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:42.068164 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:42.068277 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:42.068799 (+ 522us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:46:42.068897 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:46:42.068997 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:42.069156 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 18:46:42.069259 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 18:46:42.069602 8984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:42.060215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56020 (local address 127.30.185.190:46669)
0811 18:46:42.062599 (+ 2384us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:42.062607 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:42.062625 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:46:42.062733 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:42.062740 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:42.062960 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:42.063153 (+ 193us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:46:42.063717 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:42.064668 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:42.066990 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:42.067630 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:42.068250 (+ 620us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:42.068482 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:46:42.068929 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:46:42.069109 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:42.069218 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 18:46:42.069306 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2261,"thread_start_us":2164,"threads_started":1}
I20260811 18:46:42.079097 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 8300
I20260811 18:46:42.080166 8421 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:46:42.191701 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 8300
I20260811 18:46:42.218732 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 8443
I20260811 18:46:42.219909 8561 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:46:42.334309 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 8443
I20260811 18:46:42.363332 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 8581
I20260811 18:46:42.364543 8702 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:46:42.470597 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 8581
2026-08-11T18:46:42Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5638 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:46:42 dist-test-slave-z8x9 krb5kdc[8993](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:46:42 dist-test-slave-z8x9 krb5kdc[8993](info): set up 2 sockets
Aug 11 18:46:42 dist-test-slave-z8x9 krb5kdc[8993](info): commencing operation
krb5kdc: starting...
W20260811 18:46:43.906741 31462 mini_kdc.cc:121] Time spent starting KDC: real 1.384s user 0.003s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/test-user.keytab.
Aug 11 18:46:44 dist-test-slave-z8x9 krb5kdc[8993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:46:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:46:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.185.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
Entry for principal kudu/127.30.185.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab.
I20260811 18:46:44.273073 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:35703
--webserver_interface=127.30.185.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:42071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:35703,127.30.185.189:45943,127.30.185.188:39657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:46:44.628913 9014 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:46:44.629387 9014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:46:44.629503 9014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:46:44.629642 9014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:46:44.636899 9014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:46:44.636988 9014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:46:44.637110 9014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:46:44.637252 9014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:46:44.637355 9014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:46:44.647357 9014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:42071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:35703,127.30.185.189:45943,127.30.185.188:39657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.190.keytab
--principal=kudu/127.30.185.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.190:35703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.185.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:46:44.648746 9014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:46:44.650440 9014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:46:44.664384 9025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:44.664448 9030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:44.664451 9026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:46:44.666143 9014 server_base.cc:1034] running on GCE node
I20260811 18:46:44.667269 9014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:46:44.669297 9014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:46:44.670572 9014 hybrid_clock.cc:648] HybridClock initialized: now 1786474004670473 us; error 51 us; skew 500 ppm
I20260811 18:46:44.670907 9014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:46:44 dist-test-slave-z8x9 krb5kdc[8993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474004, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:46:44.674218 9014 init.cc:377] Logged in from keytab as kudu/127.30.185.190@KRBTEST.COM (short username kudu)
I20260811 18:46:44.675805 9014 webserver.cc:460] Webserver started at http://127.30.185.190:35165/ using document root <none> and password file <none>
I20260811 18:46:44.676592 9014 fs_manager.cc:359] Metadata directory not provided
I20260811 18:46:44.676733 9014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:46:44.677083 9014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:46:44.680215 9014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-0/data/instance:
uuid: "eae36276a70c486ba3f2c127b6803151"
format_stamp: "Formatted at 2026-08-11 18:46:44 on dist-test-slave-z8x9"
server_key: "9eeaf5c01c96cfee00ee8f64dc36c664"
server_key_iv: "fdcc6d98a46ffb99484eaa1f5eb6bc5a"
server_key_version: "encryptionkey@0"
I20260811 18:46:44.681057 9014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-0/wal/instance:
uuid: "eae36276a70c486ba3f2c127b6803151"
format_stamp: "Formatted at 2026-08-11 18:46:44 on dist-test-slave-z8x9"
server_key: "9eeaf5c01c96cfee00ee8f64dc36c664"
server_key_iv: "fdcc6d98a46ffb99484eaa1f5eb6bc5a"
server_key_version: "encryptionkey@0"
I20260811 18:46:44.686432 9014 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 18:46:44.690018 9046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:46:44.691381 9014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:46:44.691586 9014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-0/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-0/wal
uuid: "eae36276a70c486ba3f2c127b6803151"
format_stamp: "Formatted at 2026-08-11 18:46:44 on dist-test-slave-z8x9"
server_key: "9eeaf5c01c96cfee00ee8f64dc36c664"
server_key_iv: "fdcc6d98a46ffb99484eaa1f5eb6bc5a"
server_key_version: "encryptionkey@0"
I20260811 18:46:44.691836 9014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:46:44.706799 9014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:46:44.711643 9014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:46:44.711941 9014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:46:44.731016 9014 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.190:35703
I20260811 18:46:44.731040 9136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.190:35703 every 8 connection(s)
I20260811 18:46:44.732477 9014 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-0/data/info.pb
I20260811 18:46:44.736320 9138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:46:44.738306 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 9014
I20260811 18:46:44.738528 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-0/wal/instance
I20260811 18:46:44.738999 31462 external_mini_cluster.cc:1428] Setting key b4c0dfea36bce5c42ac4a54ef61cec4e
I20260811 18:46:44.743196 9138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35703 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:46:44 dist-test-slave-z8x9 krb5kdc[8993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474004, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.185.189@KRBTEST.COM; defaulting to no policy
I20260811 18:46:44.777940 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:44.746539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:35703 (local address 127.0.0.1:52992)
0811 18:46:44.756205 (+ 9666us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:44.756256 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 18:46:44.758319 (+ 2063us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:46:44.759448 (+ 1129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:46:44.759453 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:46:44.760056 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:46:44.760418 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:44.760436 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:44.762864 (+ 2428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:44.762868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:44.763966 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:44.763979 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:44.765419 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:44.765424 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:44.765622 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:44.766446 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:46:44.766482 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:46:44.768971 (+ 2489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:46:44.773573 (+ 4602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:44.773585 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:44.773595 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:44.774021 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:44.774542 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:44.774545 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:44.774548 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:44.774709 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:44.775793 (+ 1084us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:46:44.775799 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:46:44.776012 (+ 213us) client_negotiation.cc:769] Sending connection context
0811 18:46:44.776227 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 18:46:44.776640 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9387,"mutex_wait_us":53,"thread_start_us":9136,"threads_started":1}
I20260811 18:46:44.777976 9143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:44.746539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52992 (local address 127.30.185.190:35703)
0811 18:46:44.755720 (+ 9181us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:44.755745 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:44.756413 (+ 668us) server_negotiation.cc:407] Connection header received
0811 18:46:44.758793 (+ 2380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:44.758805 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:44.759161 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:44.759405 (+ 244us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:44.760652 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:44.762702 (+ 2050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:44.764119 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:44.765154 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:44.765415 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:44.769271 (+ 3856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:44.769287 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:44.769306 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:44.769359 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:44.773432 (+ 4073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:44.774164 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:44.774176 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:44.774181 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:44.774417 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:44.774821 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:44.774829 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:44.774834 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:44.775634 (+ 800us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:44.775830 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:44.776443 (+ 613us) server_negotiation.cc:299] Negotiation successful
0811 18:46:44.776843 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":80,"server-negotiator.queue_time_us":8883,"spinlock_wait_cycles":34176,"thread_start_us":8605,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.189@KRBTEST.COM" created.
I20260811 18:46:44.781041 9138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:46:44.782192 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:44.781795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:45943 (local address 127.0.0.1:37728)
0811 18:46:44.781973 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:44.782082 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260811 18:46:44.782441 9051 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.185.189:45943: connect: Connection refused (error 111)
I20260811 18:46:44.784212 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:44.783717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:45943 (local address 127.0.0.1:37730)
0811 18:46:44.783974 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:44.784056 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 18:46:44.784435 9138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.189:45943: Network error: Client connection negotiation failed: client connection to 127.30.185.189:45943: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.189.keytab.
Entry for principal kudu/127.30.185.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.189.keytab.
I20260811 18:46:44.825831 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.189:45943
--webserver_interface=127.30.185.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.189.keytab
--principal=kudu/127.30.185.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:42071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:35703,127.30.185.189:45943,127.30.185.188:39657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:46:44.833774 9138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } attempt: 1
I20260811 18:46:44.834990 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:44.834524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:45943 (local address 127.0.0.1:37738)
0811 18:46:44.834728 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:44.834858 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 18:46:44.836071 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:44.835716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:45943 (local address 127.0.0.1:37740)
0811 18:46:44.835867 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:44.835942 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 18:46:44.836248 9138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.189:45943: Network error: Client connection negotiation failed: client connection to 127.30.185.189:45943: connect: Connection refused (error 111)
I20260811 18:46:44.904580 9138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } attempt: 2
I20260811 18:46:44.905879 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:44.905406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:45943 (local address 127.0.0.1:37752)
0811 18:46:44.905644 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:44.905743 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 18:46:44.906984 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:44.906635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:45943 (local address 127.0.0.1:37766)
0811 18:46:44.906786 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:44.906855 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 18:46:44.907202 9138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.189:45943: Network error: Client connection negotiation failed: client connection to 127.30.185.189:45943: connect: Connection refused (error 111)
I20260811 18:46:44.998461 9138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } attempt: 3
I20260811 18:46:44.999691 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:44.999238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:45943 (local address 127.0.0.1:37774)
0811 18:46:44.999456 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:44.999555 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 18:46:45.000798 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.000468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:45943 (local address 127.0.0.1:37790)
0811 18:46:45.000602 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.000669 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 18:46:45.001056 9138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.189:45943: Network error: Client connection negotiation failed: client connection to 127.30.185.189:45943: connect: Connection refused (error 111)
I20260811 18:46:45.144383 9138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } attempt: 4
I20260811 18:46:45.145776 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.145221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:45943 (local address 127.0.0.1:37804)
0811 18:46:45.145550 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.145642 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189}
I20260811 18:46:45.146827 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.146507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:45943 (local address 127.0.0.1:37812)
0811 18:46:45.146637 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.146716 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.189:45943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 18:46:45.147074 9138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.189:45943: Network error: Client connection negotiation failed: client connection to 127.30.185.189:45943: connect: Connection refused (error 111)
W20260811 18:46:45.175740 9147 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:46:45.176103 9147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:46:45.176232 9147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:46:45.176378 9147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:46:45.183938 9147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:46:45.184044 9147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:46:45.184165 9147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:46:45.184250 9147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:46:45.184376 9147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:46:45.194576 9147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:42071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:35703,127.30.185.189:45943,127.30.185.188:39657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.189.keytab
--principal=kudu/127.30.185.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.189:45943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.185.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:46:45.195977 9147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:46:45.197430 9147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:46:45.209410 9159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:45.209470 9160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:45.209491 9164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:46:45.215905 9147 server_base.cc:1034] running on GCE node
I20260811 18:46:45.216799 9147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:46:45.218412 9147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:46:45.219674 9147 hybrid_clock.cc:648] HybridClock initialized: now 1786474005219610 us; error 47 us; skew 500 ppm
I20260811 18:46:45.220044 9147 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:46:45 dist-test-slave-z8x9 krb5kdc[8993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474005, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:46:45.223357 9147 init.cc:377] Logged in from keytab as kudu/127.30.185.189@KRBTEST.COM (short username kudu)
I20260811 18:46:45.224900 9147 webserver.cc:460] Webserver started at http://127.30.185.189:42229/ using document root <none> and password file <none>
I20260811 18:46:45.225669 9147 fs_manager.cc:359] Metadata directory not provided
I20260811 18:46:45.225764 9147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:46:45.226033 9147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:46:45.228931 9147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-1/data/instance:
uuid: "6d44a2fce20d475db9594a74a117d9e7"
format_stamp: "Formatted at 2026-08-11 18:46:45 on dist-test-slave-z8x9"
server_key: "e3fc656a80a5eeea448ba318f55b881a"
server_key_iv: "a74571c2bf366857929881cdc0e7c745"
server_key_version: "encryptionkey@0"
I20260811 18:46:45.229764 9147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-1/wal/instance:
uuid: "6d44a2fce20d475db9594a74a117d9e7"
format_stamp: "Formatted at 2026-08-11 18:46:45 on dist-test-slave-z8x9"
server_key: "e3fc656a80a5eeea448ba318f55b881a"
server_key_iv: "a74571c2bf366857929881cdc0e7c745"
server_key_version: "encryptionkey@0"
I20260811 18:46:45.235045 9147 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 18:46:45.238649 9173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:46:45.239966 9147 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:46:45.240114 9147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-1/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-1/wal
uuid: "6d44a2fce20d475db9594a74a117d9e7"
format_stamp: "Formatted at 2026-08-11 18:46:45 on dist-test-slave-z8x9"
server_key: "e3fc656a80a5eeea448ba318f55b881a"
server_key_iv: "a74571c2bf366857929881cdc0e7c745"
server_key_version: "encryptionkey@0"
I20260811 18:46:45.240300 9147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:46:45.253391 9147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:46:45.257231 9147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:46:45.257455 9147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:46:45.276223 9147 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.189:45943
I20260811 18:46:45.276242 9274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.189:45943 every 8 connection(s)
I20260811 18:46:45.277755 9147 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-1/data/info.pb
I20260811 18:46:45.281541 9276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:46:45.282371 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 9147
I20260811 18:46:45.282591 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-1/wal/instance
I20260811 18:46:45.283051 31462 external_mini_cluster.cc:1428] Setting key c9d64f40aa8fc4c06ea18932df71a230
I20260811 18:46:45.288338 9276 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35703 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:46:45 dist-test-slave-z8x9 krb5kdc[8993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474005, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.189@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:46:45.309749 9284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.291061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53006 (local address 127.30.185.190:35703)
0811 18:46:45.291585 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:45.291590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:45.292637 (+ 1047us) server_negotiation.cc:407] Connection header received
0811 18:46:45.294730 (+ 2093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:45.294737 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:45.294833 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:45.294969 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:45.296760 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.297615 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.299168 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.299573 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.299708 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:45.303601 (+ 3893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:45.303606 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:45.303610 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:45.303648 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:45.306527 (+ 2879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:45.307303 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:45.307309 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:45.307314 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:45.307449 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:45.307899 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:45.307906 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:45.307914 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:45.308187 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:45.308369 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:45.309090 (+ 721us) server_negotiation.cc:299] Negotiation successful
0811 18:46:45.309356 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":295,"threads_started":1}
I20260811 18:46:45.310884 9285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.291674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:35703 (local address 127.0.0.1:53006)
0811 18:46:45.292464 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.292505 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:46:45.294405 (+ 1900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:46:45.295238 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:46:45.295247 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:46:45.295874 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:46:45.296527 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:45.296549 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.297791 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.297795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:45.298998 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:45.299011 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.299834 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.299843 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:45.300067 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:45.300804 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:46:45.300831 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:46:45.303344 (+ 2513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:46:45.306677 (+ 3333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:45.306687 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:45.306704 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:45.307165 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:45.307575 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:45.307580 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:45.307584 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:45.307773 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:45.308326 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:46:45.308334 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:46:45.308726 (+ 392us) client_negotiation.cc:769] Sending connection context
0811 18:46:45.309012 (+ 286us) client_negotiation.cc:240] Negotiation successful
0811 18:46:45.309585 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":329,"threads_started":1}
I20260811 18:46:45.312704 9276 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.185.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.185.188@KRBTEST.COM" created.
Aug 11 18:46:45 dist-test-slave-z8x9 krb5kdc[8993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474005, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.189@KRBTEST.COM for kudu/127.30.185.189@KRBTEST.COM
I20260811 18:46:45.327320 9285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.313633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:45943 (local address 127.0.0.1:37822)
0811 18:46:45.313793 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.313808 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:46:45.313919 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:46:45.314996 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:46:45.315000 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:46:45.315474 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:46:45.315600 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:45.315609 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.317003 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.317015 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:45.317972 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:45.317984 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.318779 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.318784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:45.318964 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:45.319575 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:46:45.319601 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:46:45.321677 (+ 2076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:46:45.324826 (+ 3149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:45.324832 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:45.324836 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:45.325243 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:45.325628 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:45.325633 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:45.325636 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:45.325782 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:45.326360 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:46:45.326364 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:46:45.326511 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 18:46:45.326657 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 18:46:45.326935 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 18:46:45.327421 9287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.313951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37822 (local address 127.30.185.189:45943)
0811 18:46:45.314607 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:45.314612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:45.314637 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:46:45.314678 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:45.314683 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:45.314848 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:45.315003 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:45.315779 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.316839 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.318104 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.318624 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.318758 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:45.321882 (+ 3124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:45.321893 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:45.321905 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:45.321936 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:45.324671 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:45.325385 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:45.325396 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:45.325404 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:45.325513 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:45.325900 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:45.325905 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:45.325909 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:45.326238 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:45.326458 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:45.326825 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 18:46:45.327034 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":372,"threads_started":1}
I20260811 18:46:45.330029 9276 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:46:45.331061 9285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.330622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:39657 (local address 127.0.0.1:49098)
0811 18:46:45.330775 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.330912 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 18:46:45.331310 9177 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
I20260811 18:46:45.332887 9285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.332532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:39657 (local address 127.0.0.1:49104)
0811 18:46:45.332709 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.332782 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 18:46:45.333103 9276 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:39657: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.185.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.188.keytab.
Entry for principal kudu/127.30.185.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.188.keytab.
I20260811 18:46:45.359897 31462 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasky46AhN/build/asan/bin/kudu
/tmp/dist-test-tasky46AhN/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.188:39657
--webserver_interface=127.30.185.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.188.keytab
--principal=kudu/127.30.185.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.185.148:42071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.185.190:35703,127.30.185.189:45943,127.30.185.188:39657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:46:45.382522 9276 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } attempt: 1
I20260811 18:46:45.383931 9285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.383296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:39657 (local address 127.0.0.1:49108)
0811 18:46:45.383560 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.383704 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 18:46:45.385092 9285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.384578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:39657 (local address 127.0.0.1:49122)
0811 18:46:45.384785 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.384917 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 18:46:45.385334 9276 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:39657: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
I20260811 18:46:45.446352 9138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } attempt: 5
I20260811 18:46:45.453557 9276 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } attempt: 2
I20260811 18:46:45.454591 9285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.454119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:39657 (local address 127.0.0.1:49134)
0811 18:46:45.454294 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.454385 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
Aug 11 18:46:45 dist-test-slave-z8x9 krb5kdc[8993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474004, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for kudu/127.30.185.189@KRBTEST.COM
I20260811 18:46:45.456141 9285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.455707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:39657 (local address 127.0.0.1:49148)
0811 18:46:45.455885 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.455975 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 18:46:45.456763 9276 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:39657: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
I20260811 18:46:45.461989 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.447073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:45943 (local address 127.0.0.1:37836)
0811 18:46:45.447321 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.447346 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 18:46:45.447486 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:46:45.447868 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:46:45.447875 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:46:45.448502 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:46:45.448812 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:45.448830 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.449927 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.449933 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:45.451376 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:45.451394 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.452139 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.452145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:45.452385 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:45.453053 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:46:45.453076 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:46:45.455755 (+ 2679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:46:45.459687 (+ 3932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:45.459691 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:45.459694 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:45.460067 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:45.460477 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:45.460480 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:45.460482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:45.460609 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:45.461260 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:46:45.461269 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:46:45.461399 (+ 130us) client_negotiation.cc:769] Sending connection context
0811 18:46:45.461516 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 18:46:45.461744 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 18:46:45.462297 9287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.447316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37836 (local address 127.30.185.189:45943)
0811 18:46:45.447493 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:45.447497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:45.447517 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:46:45.447601 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:45.447605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:45.447679 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:45.447762 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:45.448997 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.449759 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.451536 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.451979 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.452072 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:45.456007 (+ 3935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:45.456012 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:45.456015 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:45.456046 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:45.459532 (+ 3486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:45.460200 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:45.460207 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:45.460211 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:45.460361 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:45.460716 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:45.460721 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:45.460724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:45.461108 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:45.461303 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:45.461653 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 18:46:45.461899 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 18:46:45.463466 9138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:46:45.464484 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.464057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:39657 (local address 127.0.0.1:49160)
0811 18:46:45.464226 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.464346 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 18:46:45.465438 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.465076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:39657 (local address 127.0.0.1:49172)
0811 18:46:45.465237 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.465326 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 18:46:45.465647 9138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:39657: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
I20260811 18:46:45.516894 9138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } attempt: 1
I20260811 18:46:45.518090 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.517614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:39657 (local address 127.0.0.1:49184)
0811 18:46:45.517819 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.517920 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 18:46:45.519197 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.518896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:39657 (local address 127.0.0.1:49200)
0811 18:46:45.519032 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.519100 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 18:46:45.519519 9138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:39657: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
I20260811 18:46:45.548035 9276 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } attempt: 3
I20260811 18:46:45.549376 9285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.548720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:39657 (local address 127.0.0.1:49216)
0811 18:46:45.548989 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.549138 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 18:46:45.550537 9285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.550034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:39657 (local address 127.0.0.1:49226)
0811 18:46:45.550297 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.550393 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
W20260811 18:46:45.550752 9276 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:39657: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
I20260811 18:46:45.587765 9138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } attempt: 2
I20260811 18:46:45.589092 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.588589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:39657 (local address 127.0.0.1:49230)
0811 18:46:45.588830 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.588925 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 18:46:45.590184 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.589879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:39657 (local address 127.0.0.1:49240)
0811 18:46:45.590013 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.590079 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 18:46:45.590425 9138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:39657: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
I20260811 18:46:45.694005 9276 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } attempt: 4
I20260811 18:46:45.695369 9285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.694714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:39657 (local address 127.0.0.1:49250)
0811 18:46:45.694998 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.695135 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 18:46:45.696424 9285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.696057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:39657 (local address 127.0.0.1:49258)
0811 18:46:45.696211 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.696295 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 18:46:45.696646 9276 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:39657: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
I20260811 18:46:45.696646 9138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } attempt: 3
I20260811 18:46:45.697683 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.697280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:39657 (local address 127.0.0.1:49268)
0811 18:46:45.697454 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.697535 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 18:46:45.698688 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.698324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:39657 (local address 127.0.0.1:49272)
0811 18:46:45.698506 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.698593 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260811 18:46:45.698875 9138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.185.188:39657: Network error: Client connection negotiation failed: client connection to 127.30.185.188:39657: connect: Connection refused (error 111)
W20260811 18:46:45.715036 9293 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:46:45.715428 9293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:46:45.715557 9293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:46:45.715691 9293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:46:45.722986 9293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:46:45.723115 9293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:46:45.723233 9293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:46:45.723291 9293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:46:45.723389 9293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:46:45.733180 9293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.185.148:42071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.185.190:35703,127.30.185.189:45943,127.30.185.188:39657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/krb5kdc/kudu_127.30.185.188.keytab
--principal=kudu/127.30.185.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.185.188:39657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.185.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 16:09:54 UTC on ccycloud.quasar-hitlqc.root.comops.site
build id 4580676
ASAN enabled
I20260811 18:46:45.734484 9293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:46:45.735805 9293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:46:45.748019 9302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:45.748061 9304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:46:45.748065 9307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:46:45.748996 9293 server_base.cc:1034] running on GCE node
I20260811 18:46:45.749845 9293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:46:45.751269 9293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:46:45.752480 9293 hybrid_clock.cc:648] HybridClock initialized: now 1786474005752395 us; error 55 us; skew 500 ppm
I20260811 18:46:45.752821 9293 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 18:46:45 dist-test-slave-z8x9 krb5kdc[8993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474005, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:46:45.755997 9293 init.cc:377] Logged in from keytab as kudu/127.30.185.188@KRBTEST.COM (short username kudu)
I20260811 18:46:45.757483 9293 webserver.cc:460] Webserver started at http://127.30.185.188:35691/ using document root <none> and password file <none>
I20260811 18:46:45.758188 9293 fs_manager.cc:359] Metadata directory not provided
I20260811 18:46:45.758276 9293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:46:45.758540 9293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:46:45.761507 9293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-2/data/instance:
uuid: "309290be01d24e91a83bbb6ff6ba2187"
format_stamp: "Formatted at 2026-08-11 18:46:45 on dist-test-slave-z8x9"
server_key: "5bdf0fcfc76ed2609383146fe61a7061"
server_key_iv: "356099fc65de792fe30caa04285739e1"
server_key_version: "encryptionkey@0"
I20260811 18:46:45.762250 9293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-2/wal/instance:
uuid: "309290be01d24e91a83bbb6ff6ba2187"
format_stamp: "Formatted at 2026-08-11 18:46:45 on dist-test-slave-z8x9"
server_key: "5bdf0fcfc76ed2609383146fe61a7061"
server_key_iv: "356099fc65de792fe30caa04285739e1"
server_key_version: "encryptionkey@0"
I20260811 18:46:45.767479 9293 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 18:46:45.770802 9321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:46:45.772140 9293 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:46:45.772328 9293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-2/data,/tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-2/wal
uuid: "309290be01d24e91a83bbb6ff6ba2187"
format_stamp: "Formatted at 2026-08-11 18:46:45 on dist-test-slave-z8x9"
server_key: "5bdf0fcfc76ed2609383146fe61a7061"
server_key_iv: "356099fc65de792fe30caa04285739e1"
server_key_version: "encryptionkey@0"
I20260811 18:46:45.772557 9293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:46:45.794279 9293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:46:45.798858 9293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:46:45.799199 9293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:46:45.818228 9293 rpc_server.cc:307] RPC server started. Bound to: 127.30.185.188:39657
I20260811 18:46:45.818234 9423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.185.188:39657 every 8 connection(s)
I20260811 18:46:45.819691 9293 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-2/data/info.pb
I20260811 18:46:45.823514 9425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:46:45.825645 31462 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasky46AhN/build/asan/bin/kudu as pid 9293
I20260811 18:46:45.825834 31462 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasky46AhN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786473610573616-31462-0/minicluster-data/master-2/wal/instance
I20260811 18:46:45.826416 31462 external_mini_cluster.cc:1428] Setting key 71f525e5ed44f84ab9a93e45cc305a4b
I20260811 18:46:45.832825 9425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35703 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:46:45 dist-test-slave-z8x9 krb5kdc[8993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.188@KRBTEST.COM
Aug 11 18:46:45 dist-test-slave-z8x9 krb5kdc[8993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474005, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.188@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:46:45.850684 9430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.828398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49286 (local address 127.30.185.188:39657)
0811 18:46:45.829264 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:45.829276 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:45.829319 (+ 43us) server_negotiation.cc:407] Connection header received
0811 18:46:45.830893 (+ 1574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:45.830919 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:45.831320 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:45.831685 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:45.833227 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.834552 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.836421 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.837119 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.837335 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:45.842095 (+ 4760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:45.842106 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:45.842120 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:45.842171 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:45.845949 (+ 3778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:45.846670 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:45.846682 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:45.846692 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:45.846902 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:45.847327 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:45.847332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:45.847335 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:45.848108 (+ 773us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:45.848444 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:45.848837 (+ 393us) server_negotiation.cc:299] Negotiation successful
0811 18:46:45.849329 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":344,"threads_started":1}
I20260811 18:46:45.852180 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.835315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.190:35703 (local address 127.0.0.1:53008)
0811 18:46:45.836071 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.836114 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:46:45.836258 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:46:45.836976 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:46:45.836983 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:46:45.837951 (+ 968us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:46:45.838309 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:45.838322 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.839384 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.839390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:45.840775 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:45.840793 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.842006 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.842014 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:45.842213 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:45.842942 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:46:45.842995 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:46:45.845954 (+ 2959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:46:45.849696 (+ 3742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:45.849705 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:45.849710 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:45.850120 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:45.850524 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:45.850527 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:45.850529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:45.850661 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:45.851265 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:46:45.851271 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:46:45.851442 (+ 171us) client_negotiation.cc:769] Sending connection context
0811 18:46:45.851600 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 18:46:45.851822 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":388,"threads_started":1}
I20260811 18:46:45.852416 9434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.835342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53008 (local address 127.30.185.190:35703)
0811 18:46:45.836639 (+ 1297us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:45.836643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:45.836658 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:46:45.836708 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:45.836711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:45.836776 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:45.836908 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:45.838481 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.839212 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.840984 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.841747 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.841889 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:45.846181 (+ 4292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:45.846186 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:45.846190 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:45.846223 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:45.849568 (+ 3345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:45.850245 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:45.850251 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:45.850254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:45.850413 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:45.850780 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:45.850786 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:45.850788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:45.851139 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:45.851305 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:45.851740 (+ 435us) server_negotiation.cc:299] Negotiation successful
0811 18:46:45.851980 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1202,"thread_start_us":1088,"threads_started":1}
I20260811 18:46:45.853897 9425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:46:45 dist-test-slave-z8x9 krb5kdc[8993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474005, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.188@KRBTEST.COM for kudu/127.30.185.189@KRBTEST.COM
Aug 11 18:46:45 dist-test-slave-z8x9 krb5kdc[8993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.190@KRBTEST.COM
I20260811 18:46:45.869899 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.854758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.189:45943 (local address 127.0.0.1:37850)
0811 18:46:45.855024 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.855046 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:46:45.855161 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:46:45.855477 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:46:45.855482 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:46:45.855962 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:46:45.856157 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:45.856170 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.857606 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.857611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:45.859047 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:45.859064 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.860055 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.860061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:45.860252 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:45.860971 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:46:45.860997 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:46:45.863743 (+ 2746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:46:45.867226 (+ 3483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:45.867232 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:45.867236 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:45.867748 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:45.868106 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:45.868114 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:45.868119 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:45.868293 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:45.868937 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:46:45.868941 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:46:45.869110 (+ 169us) client_negotiation.cc:769] Sending connection context
0811 18:46:45.869299 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 18:46:45.869576 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157}
I20260811 18:46:45.870014 9287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.854723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37850 (local address 127.30.185.189:45943)
0811 18:46:45.854915 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:45.854919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:45.855120 (+ 201us) server_negotiation.cc:407] Connection header received
0811 18:46:45.855266 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:45.855270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:45.855326 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:45.855411 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:45.856306 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.857445 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.859225 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.859904 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.860018 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:45.864002 (+ 3984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:45.864008 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:45.864012 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:45.864043 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:45.867088 (+ 3045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:45.867864 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:45.867868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:45.867870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:45.867994 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:45.868439 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:45.868446 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:45.868451 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:45.868804 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:45.868978 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:45.869408 (+ 430us) server_negotiation.cc:299] Negotiation successful
0811 18:46:45.869642 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 18:46:45.871037 9425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:46:45.871312 9434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.854146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53012 (local address 127.30.185.190:35703)
0811 18:46:45.854377 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:45.854382 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:45.854400 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:46:45.854530 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:45.854535 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:45.854649 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:45.854745 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:45.856404 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.857489 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.859224 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.859887 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.860018 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:45.865236 (+ 5218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:45.865242 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:45.865247 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:45.865290 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:45.868317 (+ 3027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:45.869121 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:45.869126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:45.869130 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:45.869329 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:45.869766 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:45.869771 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:45.869775 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:45.870145 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:45.870400 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:45.870777 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 18:46:45.870996 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260811 18:46:45.876082 9138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } attempt: 4
Aug 11 18:46:45 dist-test-slave-z8x9 krb5kdc[8993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474005, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.188@KRBTEST.COM for kudu/127.30.185.188@KRBTEST.COM
Aug 11 18:46:45 dist-test-slave-z8x9 krb5kdc[8993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.185.189@KRBTEST.COM
I20260811 18:46:45.887131 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.871785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:39657 (local address 127.0.0.1:49302)
0811 18:46:45.872045 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.872066 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:46:45.872183 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:46:45.872573 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:46:45.872576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:46:45.873034 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:46:45.873247 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:45.873259 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.874628 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.874632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:45.875802 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:45.875817 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.876865 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.876870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:45.877109 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:45.877857 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:46:45.877876 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:46:45.880359 (+ 2483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:46:45.884210 (+ 3851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:45.884218 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:45.884221 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:45.884809 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:45.885280 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:45.885283 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:45.885285 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:45.885408 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:45.885961 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:46:45.885966 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:46:45.886170 (+ 204us) client_negotiation.cc:769] Sending connection context
0811 18:46:45.886326 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 18:46:45.886674 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260811 18:46:45.887820 9430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.871865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49302 (local address 127.30.185.188:39657)
0811 18:46:45.872025 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:45.872029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:45.872120 (+ 91us) server_negotiation.cc:407] Connection header received
0811 18:46:45.872313 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:45.872318 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:45.872408 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:45.872538 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:45.873400 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.874492 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.875941 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.876572 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.876699 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:45.880553 (+ 3854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:45.880559 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:45.880564 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:45.880599 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:45.884065 (+ 3466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:45.884942 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:45.884948 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:45.884953 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:45.885104 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:45.885516 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:45.885522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:45.885525 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:45.885793 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:45.885985 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:45.886493 (+ 508us) server_negotiation.cc:299] Negotiation successful
0811 18:46:45.886867 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":98,"server-negotiator.queue_time_us":58}
Aug 11 18:46:45 dist-test-slave-z8x9 krb5kdc[8993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474004, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.190@KRBTEST.COM for kudu/127.30.185.188@KRBTEST.COM
I20260811 18:46:45.892010 9287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.873316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37854 (local address 127.30.185.189:45943)
0811 18:46:45.873472 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:45.873476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:45.873489 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:46:45.873525 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:45.873528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:45.873575 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:45.873643 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:45.875112 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.876159 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.878318 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.878991 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.879145 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:45.885102 (+ 5957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:45.885110 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:45.885115 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:45.885155 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:45.888821 (+ 3666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:45.889716 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:45.889725 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:45.889732 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:45.889922 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:45.890447 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:45.890453 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:45.890456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:45.890833 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:45.891118 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:45.891426 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 18:46:45.891674 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:46:45.895038 9425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 309290be01d24e91a83bbb6ff6ba2187: Bootstrap starting.
I20260811 18:46:45.897317 31462 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:46:45.898204 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.876675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:39657 (local address 127.0.0.1:49308)
0811 18:46:45.876872 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.876891 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:46:45.876990 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:46:45.882725 (+ 5735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:46:45.882733 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:46:45.883181 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:46:45.883342 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:45.883381 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.884815 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.884821 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:45.886326 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:45.886344 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.887335 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.887342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:45.887546 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:45.888091 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:46:45.888109 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:46:45.890611 (+ 2502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:46:45.895093 (+ 4482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:45.895098 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:45.895102 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:45.895669 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:45.896216 (+ 547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:45.896224 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:45.896228 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:45.896397 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:45.897083 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:46:45.897088 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:46:45.897358 (+ 270us) client_negotiation.cc:769] Sending connection context
0811 18:46:45.897532 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 18:46:45.897826 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 18:46:45.898524 9436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.876994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49308 (local address 127.30.185.188:39657)
0811 18:46:45.882437 (+ 5443us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:45.882442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:45.882461 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:46:45.882498 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:45.882502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:45.882554 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:45.882693 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:45.883556 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.884675 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.886493 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.887152 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.887845 (+ 693us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:45.890862 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:45.890869 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:45.890874 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:45.890910 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:45.894948 (+ 4038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:45.895916 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:45.895921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:45.895924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:45.896108 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:45.896517 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:45.896523 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:45.896526 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:45.896928 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:45.897113 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:45.897629 (+ 516us) server_negotiation.cc:299] Negotiation successful
0811 18:46:45.897837 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5312,"thread_start_us":5075,"threads_started":1}
I20260811 18:46:45.899296 9425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 309290be01d24e91a83bbb6ff6ba2187: Neither blocks nor log segments found. Creating new log.
I20260811 18:46:45.900709 9425 log.cc:824] T 00000000000000000000000000000000 P 309290be01d24e91a83bbb6ff6ba2187: Log is configured to *not* fsync() on all Append() calls
I20260811 18:46:45.903910 9138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151: Bootstrap starting.
I20260811 18:46:45.904136 9425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 309290be01d24e91a83bbb6ff6ba2187: No bootstrap required, opened a new log
I20260811 18:46:45.900419 9095 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "eae36276a70c486ba3f2c127b6803151"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53012
I20260811 18:46:45.907027 9138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151: Neither blocks nor log segments found. Creating new log.
I20260811 18:46:45.908017 9138 log.cc:824] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151: Log is configured to *not* fsync() on all Append() calls
I20260811 18:46:45.911257 9138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151: No bootstrap required, opened a new log
I20260811 18:46:45.913053 9425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 309290be01d24e91a83bbb6ff6ba2187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae36276a70c486ba3f2c127b6803151" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35703 } } peers { permanent_uuid: "6d44a2fce20d475db9594a74a117d9e7" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } } peers { permanent_uuid: "309290be01d24e91a83bbb6ff6ba2187" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } }
I20260811 18:46:45.913483 9425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 309290be01d24e91a83bbb6ff6ba2187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 309290be01d24e91a83bbb6ff6ba2187, State: Initialized, Role: FOLLOWER
I20260811 18:46:45.913928 9425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 309290be01d24e91a83bbb6ff6ba2187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae36276a70c486ba3f2c127b6803151" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35703 } } peers { permanent_uuid: "6d44a2fce20d475db9594a74a117d9e7" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } } peers { permanent_uuid: "309290be01d24e91a83bbb6ff6ba2187" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } }
I20260811 18:46:45.914860 9442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 309290be01d24e91a83bbb6ff6ba2187 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae36276a70c486ba3f2c127b6803151" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35703 } } peers { permanent_uuid: "6d44a2fce20d475db9594a74a117d9e7" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } } peers { permanent_uuid: "309290be01d24e91a83bbb6ff6ba2187" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } } }
I20260811 18:46:45.915256 9442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 309290be01d24e91a83bbb6ff6ba2187 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:46:45.915979 9425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 309290be01d24e91a83bbb6ff6ba2187 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:46:45.918376 9138 raft_consensus.cc:348] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae36276a70c486ba3f2c127b6803151" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35703 } } peers { permanent_uuid: "6d44a2fce20d475db9594a74a117d9e7" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } } peers { permanent_uuid: "309290be01d24e91a83bbb6ff6ba2187" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } }
I20260811 18:46:45.918797 9138 raft_consensus.cc:729] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eae36276a70c486ba3f2c127b6803151, State: Initialized, Role: FOLLOWER
I20260811 18:46:45.919281 9138 consensus_queue.cc:260] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae36276a70c486ba3f2c127b6803151" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35703 } } peers { permanent_uuid: "6d44a2fce20d475db9594a74a117d9e7" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } } peers { permanent_uuid: "309290be01d24e91a83bbb6ff6ba2187" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } }
I20260811 18:46:45.920436 9458 sys_catalog.cc:455] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae36276a70c486ba3f2c127b6803151" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35703 } } peers { permanent_uuid: "6d44a2fce20d475db9594a74a117d9e7" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } } peers { permanent_uuid: "309290be01d24e91a83bbb6ff6ba2187" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } } }
I20260811 18:46:45.920974 9458 sys_catalog.cc:458] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:46:45.921562 9138 sys_catalog.cc:564] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:46:45.921689 9095 raft_consensus.cc:482] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 18:46:45.921950 9095 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:46:45.926852 9095 raft_consensus.cc:504] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae36276a70c486ba3f2c127b6803151" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35703 } } peers { permanent_uuid: "6d44a2fce20d475db9594a74a117d9e7" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } } peers { permanent_uuid: "309290be01d24e91a83bbb6ff6ba2187" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } }
I20260811 18:46:45.929911 9095 leader_election.cc:288] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151 [CANDIDATE]: Term 1 election: Requested vote from peers 6d44a2fce20d475db9594a74a117d9e7 (127.30.185.189:45943), 309290be01d24e91a83bbb6ff6ba2187 (127.30.185.188:39657)
I20260811 18:46:45.930622 9372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eae36276a70c486ba3f2c127b6803151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "309290be01d24e91a83bbb6ff6ba2187"
I20260811 18:46:45.931118 9372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 309290be01d24e91a83bbb6ff6ba2187 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:46:45.935623 9372 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 309290be01d24e91a83bbb6ff6ba2187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eae36276a70c486ba3f2c127b6803151 in term 1.
I20260811 18:46:45.936596 9049 leader_election.cc:302] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 309290be01d24e91a83bbb6ff6ba2187, eae36276a70c486ba3f2c127b6803151; no voters:
I20260811 18:46:45.937283 9458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:46:45.939499 9458 raft_consensus.cc:686] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151 [term 1 LEADER]: Becoming Leader. State: Replica: eae36276a70c486ba3f2c127b6803151, State: Running, Role: LEADER
I20260811 18:46:45.940245 9458 consensus_queue.cc:237] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae36276a70c486ba3f2c127b6803151" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35703 } } peers { permanent_uuid: "6d44a2fce20d475db9594a74a117d9e7" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } } peers { permanent_uuid: "309290be01d24e91a83bbb6ff6ba2187" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } }
I20260811 18:46:45.945859 9493 sys_catalog.cc:455] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eae36276a70c486ba3f2c127b6803151. Latest consensus state: current_term: 1 leader_uuid: "eae36276a70c486ba3f2c127b6803151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae36276a70c486ba3f2c127b6803151" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35703 } } peers { permanent_uuid: "6d44a2fce20d475db9594a74a117d9e7" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } } peers { permanent_uuid: "309290be01d24e91a83bbb6ff6ba2187" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } } }
I20260811 18:46:45.947028 9493 sys_catalog.cc:458] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:45.941888 9014 master_runner.cc:186] Error getting master registration for 127.30.185.189:45943: OK, instance_id { permanent_uuid: "6d44a2fce20d475db9594a74a117d9e7" instance_seqno: 1786474005270763 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:46:45.953912 9014 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:46:45.947618 9293 master_runner.cc:186] Error getting master registration for 127.30.185.189:45943: OK, instance_id { permanent_uuid: "6d44a2fce20d475db9594a74a117d9e7" instance_seqno: 1786474005270763 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:46:45.960741 9293 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 18:46:45.963630 9484 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 309290be01d24e91a83bbb6ff6ba2187: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:46:45.963781 9484 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 18:46:45.964926 9057 tablet.cc:2369] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:46:45.971769 9485 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:46:45.971930 9485 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:46:45.972055 9503 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:46:45.974274 9503 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:46:45.983978 9372 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 309290be01d24e91a83bbb6ff6ba2187 [term 1 FOLLOWER]: Refusing update from remote peer eae36276a70c486ba3f2c127b6803151: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:46:45.985430 9493 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "309290be01d24e91a83bbb6ff6ba2187" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:46:45.995836 9276 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } attempt: 5
I20260811 18:46:45.999410 9434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.970980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53024 (local address 127.30.185.190:35703)
0811 18:46:45.971213 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:45.971220 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:45.971800 (+ 580us) server_negotiation.cc:407] Connection header received
0811 18:46:45.972139 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:45.972145 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:45.972230 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:45.972317 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:45.975125 (+ 2808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.976459 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.979715 (+ 3256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.980745 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:45.980894 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:45.989050 (+ 8156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:45.989058 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:45.989063 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:45.989112 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:45.993295 (+ 4183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:45.994638 (+ 1343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:45.994651 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:45.994656 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:45.995004 (+ 348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:45.995838 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:45.995871 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:45.995875 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:45.996677 (+ 802us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:45.997269 (+ 592us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:45.998283 (+ 1014us) server_negotiation.cc:299] Negotiation successful
0811 18:46:45.998695 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 18:46:46 dist-test-slave-z8x9 krb5kdc[8993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474005, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.185.189@KRBTEST.COM for kudu/127.30.185.188@KRBTEST.COM
I20260811 18:46:46.022040 9442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 309290be01d24e91a83bbb6ff6ba2187 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eae36276a70c486ba3f2c127b6803151. Latest consensus state: current_term: 1 leader_uuid: "eae36276a70c486ba3f2c127b6803151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae36276a70c486ba3f2c127b6803151" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35703 } } peers { permanent_uuid: "6d44a2fce20d475db9594a74a117d9e7" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } } peers { permanent_uuid: "309290be01d24e91a83bbb6ff6ba2187" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } } }
I20260811 18:46:46.022384 9442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 309290be01d24e91a83bbb6ff6ba2187 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:46:46.023623 9285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.996435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.185.188:39657 (local address 127.0.0.1:49326)
0811 18:46:45.996820 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0811 18:46:45.996842 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:46:45.996963 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:46:45.997891 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:46:45.997897 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:46:45.998439 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:46:45.998658 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:45.998672 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:46.000192 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:46.000198 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:46.001686 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:46:46.001703 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:46:46.003194 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:46:46.003201 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:46:46.003436 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:46.004317 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:46:46.004342 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:46:46.008686 (+ 4344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:46:46.017682 (+ 8996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:46.017690 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:46.017696 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:46.018235 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:46.018761 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:46:46.018766 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:46:46.018797 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:46:46.019239 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:46:46.021266 (+ 2027us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:46:46.021276 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:46:46.021634 (+ 358us) client_negotiation.cc:769] Sending connection context
0811 18:46:46.021931 (+ 297us) client_negotiation.cc:240] Negotiation successful
0811 18:46:46.022695 (+ 764us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256}
I20260811 18:46:46.025873 9430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.997216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49326 (local address 127.30.185.188:39657)
0811 18:46:45.997419 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:45.997427 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:45.997445 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:46:45.997496 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:45.997501 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:45.997574 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:45.997671 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:45.998837 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.999970 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:46.002019 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:46.002787 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:46.002926 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:46.008997 (+ 6071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:46.009007 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:46.009015 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:46.009071 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:46.017502 (+ 8431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:46.018384 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:46.018389 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:46.018393 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:46.018565 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:46.019548 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:46.019557 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:46.019599 (+ 42us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:46.020910 (+ 1311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:46.024098 (+ 3188us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:46.024386 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 18:46:46.024878 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64,"spinlock_wait_cycles":1280}
I20260811 18:46:46.027725 9436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.972253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49320 (local address 127.30.185.188:39657)
0811 18:46:45.972488 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:45.972493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:45.987113 (+ 14620us) server_negotiation.cc:407] Connection header received
0811 18:46:45.988430 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:45.988439 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:45.988542 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:45.988627 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:45.993291 (+ 4664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.994497 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:46.000290 (+ 5793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:46.001015 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:46.001122 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:46.012027 (+ 10905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:46.012033 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:46.012038 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:46.012084 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:46.016057 (+ 3973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:46.019592 (+ 3535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:46.019597 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:46.019600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:46.019914 (+ 314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:46.025511 (+ 5597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:46.025517 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:46.025520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:46.025927 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:46.026455 (+ 528us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:46.027069 (+ 614us) server_negotiation.cc:299] Negotiation successful
0811 18:46:46.027306 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"spinlock_wait_cycles":4992}
I20260811 18:46:46.031946 9276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d44a2fce20d475db9594a74a117d9e7: Bootstrap starting.
I20260811 18:46:46.032140 9287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:45.976184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37860 (local address 127.30.185.189:45943)
0811 18:46:45.976367 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:45.976386 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:45.991867 (+ 15481us) server_negotiation.cc:407] Connection header received
0811 18:46:45.992315 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:45.992321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:45.992410 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:45.992482 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:45.997442 (+ 4960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:45.998546 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:46.007186 (+ 8640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:46.007924 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:46.008035 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:46.019962 (+ 11927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:46:46.019972 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:46:46.019981 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:46:46.020022 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:46:46.023922 (+ 3900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:46.029288 (+ 5366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:46.029293 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:46.029297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:46.029478 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:46:46.030032 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:46:46.030038 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:46:46.030042 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:46:46.030427 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:46:46.030672 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:46.031288 (+ 616us) server_negotiation.cc:299] Negotiation successful
0811 18:46:46.031662 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 18:46:46.035434 9519 mvcc.cc:205] Tried to move back new op lower bound from 7317397528497291264 to 7317397528337489920. Current Snapshot: MvccSnapshot[applied={T|T < 7317397528497291264}]
I20260811 18:46:46.036507 9276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d44a2fce20d475db9594a74a117d9e7: Neither blocks nor log segments found. Creating new log.
I20260811 18:46:46.037853 9276 log.cc:824] T 00000000000000000000000000000000 P 6d44a2fce20d475db9594a74a117d9e7: Log is configured to *not* fsync() on all Append() calls
I20260811 18:46:46.041477 9276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d44a2fce20d475db9594a74a117d9e7: No bootstrap required, opened a new log
I20260811 18:46:46.049846 9442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 309290be01d24e91a83bbb6ff6ba2187 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eae36276a70c486ba3f2c127b6803151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae36276a70c486ba3f2c127b6803151" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35703 } } peers { permanent_uuid: "6d44a2fce20d475db9594a74a117d9e7" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } } peers { permanent_uuid: "309290be01d24e91a83bbb6ff6ba2187" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } } }
I20260811 18:46:46.050113 9442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 309290be01d24e91a83bbb6ff6ba2187 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:46:46.051754 9493 sys_catalog.cc:455] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eae36276a70c486ba3f2c127b6803151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae36276a70c486ba3f2c127b6803151" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35703 } } peers { permanent_uuid: "6d44a2fce20d475db9594a74a117d9e7" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } } peers { permanent_uuid: "309290be01d24e91a83bbb6ff6ba2187" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } } }
I20260811 18:46:46.052047 9493 sys_catalog.cc:458] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:46.052858 9276 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d44a2fce20d475db9594a74a117d9e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae36276a70c486ba3f2c127b6803151" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35703 } } peers { permanent_uuid: "6d44a2fce20d475db9594a74a117d9e7" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } } peers { permanent_uuid: "309290be01d24e91a83bbb6ff6ba2187" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } }
I20260811 18:46:46.053313 9276 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d44a2fce20d475db9594a74a117d9e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d44a2fce20d475db9594a74a117d9e7, State: Initialized, Role: FOLLOWER
I20260811 18:46:46.053826 9276 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d44a2fce20d475db9594a74a117d9e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae36276a70c486ba3f2c127b6803151" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35703 } } peers { permanent_uuid: "6d44a2fce20d475db9594a74a117d9e7" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } } peers { permanent_uuid: "309290be01d24e91a83bbb6ff6ba2187" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } }
I20260811 18:46:46.059043 9503 catalog_manager.cc:1347] Generated new cluster ID: 86430b8f9cb04fa1820db888fee6357e
I20260811 18:46:46.059157 9503 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:46:46.060842 9529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d44a2fce20d475db9594a74a117d9e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae36276a70c486ba3f2c127b6803151" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35703 } } peers { permanent_uuid: "6d44a2fce20d475db9594a74a117d9e7" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } } peers { permanent_uuid: "309290be01d24e91a83bbb6ff6ba2187" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } } }
I20260811 18:46:46.061281 9529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d44a2fce20d475db9594a74a117d9e7 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:46:46.063024 9276 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d44a2fce20d475db9594a74a117d9e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:46:46.066244 9222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d44a2fce20d475db9594a74a117d9e7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:46:46.069375 9225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eae36276a70c486ba3f2c127b6803151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6d44a2fce20d475db9594a74a117d9e7"
I20260811 18:46:46.074429 9222 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6d44a2fce20d475db9594a74a117d9e7 [term 1 FOLLOWER]: Refusing update from remote peer eae36276a70c486ba3f2c127b6803151: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:46:46.084573 9225 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6d44a2fce20d475db9594a74a117d9e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eae36276a70c486ba3f2c127b6803151 in term 1.
I20260811 18:46:46.087975 9458 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d44a2fce20d475db9594a74a117d9e7" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:46:46.112574 9503 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:46:46.114598 9503 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:46:46.130679 9503 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151: Generated new TSK 0
I20260811 18:46:46.137650 9503 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:46:46.172811 9529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d44a2fce20d475db9594a74a117d9e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eae36276a70c486ba3f2c127b6803151. Latest consensus state: current_term: 1 leader_uuid: "eae36276a70c486ba3f2c127b6803151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae36276a70c486ba3f2c127b6803151" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35703 } } peers { permanent_uuid: "6d44a2fce20d475db9594a74a117d9e7" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } } peers { permanent_uuid: "309290be01d24e91a83bbb6ff6ba2187" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } } }
I20260811 18:46:46.173197 9529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d44a2fce20d475db9594a74a117d9e7 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:46:46.173620 9458 sys_catalog.cc:455] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eae36276a70c486ba3f2c127b6803151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae36276a70c486ba3f2c127b6803151" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35703 } } peers { permanent_uuid: "6d44a2fce20d475db9594a74a117d9e7" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } } peers { permanent_uuid: "309290be01d24e91a83bbb6ff6ba2187" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } } }
I20260811 18:46:46.173604 9550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d44a2fce20d475db9594a74a117d9e7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eae36276a70c486ba3f2c127b6803151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eae36276a70c486ba3f2c127b6803151" member_type: VOTER last_known_addr { host: "127.30.185.190" port: 35703 } } peers { permanent_uuid: "6d44a2fce20d475db9594a74a117d9e7" member_type: VOTER last_known_addr { host: "127.30.185.189" port: 45943 } } peers { permanent_uuid: "309290be01d24e91a83bbb6ff6ba2187" member_type: VOTER last_known_addr { host: "127.30.185.188" port: 39657 } } }
I20260811 18:46:46.173967 9550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d44a2fce20d475db9594a74a117d9e7 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:46:46.173974 9458 sys_catalog.cc:458] T 00000000000000000000000000000000 P eae36276a70c486ba3f2c127b6803151 [sys.catalog]: This master's current role is: LEADER
I20260811 18:46:46.181360 9564 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260811 18:46:46.251863 9434 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:46.241881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53032 (local address 127.30.185.190:35703)
0811 18:46:46.242330 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:46.242339 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:46.249819 (+ 7480us) server_negotiation.cc:407] Connection header received
0811 18:46:46.250129 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:46.250136 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:46.250242 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:46.250319 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:46.251598 (+ 1279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53032: BlockingRecv error: recv got EOF from 127.0.0.1:53032 (error 108)
Metrics: {"server-negotiator.queue_time_us":290}
W20260811 18:46:46.254230 9287 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:46.241623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37868 (local address 127.30.185.189:45943)
0811 18:46:46.242140 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:46.242149 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:46.252561 (+ 10412us) server_negotiation.cc:407] Connection header received
0811 18:46:46.252809 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:46.252815 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:46.252911 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:46.252983 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:46.253998 (+ 1015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37868: BlockingRecv error: recv got EOF from 127.0.0.1:37868 (error 108)
Metrics: {"server-negotiator.queue_time_us":340}
W20260811 18:46:46.256359 9436 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:46.250132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49334 (local address 127.30.185.188:39657)
0811 18:46:46.250360 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:46.250368 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:46.254739 (+ 4371us) server_negotiation.cc:407] Connection header received
0811 18:46:46.254987 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:46.254993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:46.255082 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:46.255158 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:46.256091 (+ 933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49334: BlockingRecv error: recv got EOF from 127.0.0.1:49334 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 18:46:46.266508 9287 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:46.264287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37876 (local address 127.30.185.189:45943)
0811 18:46:46.264504 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:46.264514 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:46.265034 (+ 520us) server_negotiation.cc:407] Connection header received
0811 18:46:46.265319 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:46.265331 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:46.265422 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:46.265533 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:46.266304 (+ 771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37876: BlockingRecv error: recv got EOF from 127.0.0.1:37876 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 18:46:46.273280 9287 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:46.271598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37884 (local address 127.30.185.189:45943)
0811 18:46:46.271770 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:46.271776 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:46.271897 (+ 121us) server_negotiation.cc:407] Connection header received
0811 18:46:46.272146 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:46.272151 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:46.272242 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:46.272347 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:46.273024 (+ 677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37884: BlockingRecv error: recv got EOF from 127.0.0.1:37884 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 18:46:46.323269 9287 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:46.317950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37898 (local address 127.30.185.189:45943)
0811 18:46:46.318142 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:46.318149 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:46.321662 (+ 3513us) server_negotiation.cc:407] Connection header received
0811 18:46:46.321946 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:46.321953 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:46.322054 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:46.322169 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:46.323018 (+ 849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37898: BlockingRecv error: recv got EOF from 127.0.0.1:37898 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 18:46:46.333590 9287 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:46.331495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37906 (local address 127.30.185.189:45943)
0811 18:46:46.331743 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:46.331752 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:46.332129 (+ 377us) server_negotiation.cc:407] Connection header received
0811 18:46:46.332366 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:46.332372 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:46.332470 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:46.332579 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:46.333364 (+ 785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37906: BlockingRecv error: recv got EOF from 127.0.0.1:37906 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260811 18:46:46.348008 9287 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:46.345741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37918 (local address 127.30.185.189:45943)
0811 18:46:46.345986 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:46.345996 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:46.346565 (+ 569us) server_negotiation.cc:407] Connection header received
0811 18:46:46.346803 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:46.346809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:46.346899 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:46.347008 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:46.347762 (+ 754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37918: BlockingRecv error: recv got EOF from 127.0.0.1:37918 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 18:46:46.372900 9287 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:46.370621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37922 (local address 127.30.185.189:45943)
0811 18:46:46.370855 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:46.370864 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:46.371426 (+ 562us) server_negotiation.cc:407] Connection header received
0811 18:46:46.371659 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:46.371664 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:46.371747 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:46.371869 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:46.372653 (+ 784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37922: BlockingRecv error: recv got EOF from 127.0.0.1:37922 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 18:46:46.410808 9287 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:46.408374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37926 (local address 127.30.185.189:45943)
0811 18:46:46.408608 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:46.408617 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:46.409223 (+ 606us) server_negotiation.cc:407] Connection header received
0811 18:46:46.409467 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:46.409473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:46.409574 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:46.409683 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:46.410533 (+ 850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37926: BlockingRecv error: recv got EOF from 127.0.0.1:37926 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 18:46:46.480870 9287 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:46.478555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37928 (local address 127.30.185.189:45943)
0811 18:46:46.478797 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:46.478807 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:46.479396 (+ 589us) server_negotiation.cc:407] Connection header received
0811 18:46:46.479651 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:46.479655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:46.479732 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:46.479858 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:46.480639 (+ 781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37928: BlockingRecv error: recv got EOF from 127.0.0.1:37928 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 18:46:46.618065 9287 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:46.615515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37944 (local address 127.30.185.189:45943)
0811 18:46:46.615758 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:46.615764 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:46.616154 (+ 390us) server_negotiation.cc:407] Connection header received
0811 18:46:46.616398 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:46.616402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:46.616513 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:46.616611 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:46.617731 (+ 1120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37944: BlockingRecv error: recv got EOF from 127.0.0.1:37944 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260811 18:46:46.881126 9287 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:46:46.879143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37946 (local address 127.30.185.189:45943)
0811 18:46:46.879443 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:46.879452 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:46.879636 (+ 184us) server_negotiation.cc:407] Connection header received
0811 18:46:46.879840 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:46.879844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:46.879919 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:46.880012 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:46:46.880898 (+ 886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37946: BlockingRecv error: recv got EOF from 127.0.0.1:37946 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 18:46:46.965855 9484 catalog_manager.cc:1259] Loaded cluster ID: 86430b8f9cb04fa1820db888fee6357e
I20260811 18:46:46.965973 9484 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 309290be01d24e91a83bbb6ff6ba2187: loading cluster ID for follower catalog manager: success
I20260811 18:46:46.968873 9484 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 309290be01d24e91a83bbb6ff6ba2187: acquiring CA information for follower catalog manager: success
I20260811 18:46:46.970854 9484 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 309290be01d24e91a83bbb6ff6ba2187: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:46:47.185804 9564 catalog_manager.cc:1259] Loaded cluster ID: 86430b8f9cb04fa1820db888fee6357e
I20260811 18:46:47.185994 9564 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6d44a2fce20d475db9594a74a117d9e7: loading cluster ID for follower catalog manager: success
I20260811 18:46:47.188594 9564 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6d44a2fce20d475db9594a74a117d9e7: acquiring CA information for follower catalog manager: success
I20260811 18:46:47.190526 9564 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6d44a2fce20d475db9594a74a117d9e7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:46:47.406282 9698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:47.398407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37952 (local address 127.30.185.189:45943)
0811 18:46:47.399275 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:47.399279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:47.399298 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:46:47.399363 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:47.399368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:47.399525 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:47.399646 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:46:47.400270 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:47.401139 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:47.403253 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:47.403756 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:47.403874 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:47.404555 (+ 681us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:46:47.405299 (+ 744us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:46:47.405423 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:47.405823 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 18:46:47.405929 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":596,"threads_started":1}
I20260811 18:46:47.421442 9708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:47.413254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49350 (local address 127.30.185.188:39657)
0811 18:46:47.415523 (+ 2269us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:47.415528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:47.415545 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:46:47.415642 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:47.415646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:47.415804 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:47.416048 (+ 244us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:46:47.416559 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:47.417522 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:47.419025 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:47.419474 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:47.419673 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:47.420251 (+ 578us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:46:47.420789 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:46:47.420959 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:47.421019 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 18:46:47.421102 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2162,"thread_start_us":2077,"threads_started":1}
I20260811 18:46:47.437879 9708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:47.429592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49352 (local address 127.30.185.188:39657)
0811 18:46:47.429785 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:47.429792 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:47.429823 (+ 31us) server_negotiation.cc:407] Connection header received
0811 18:46:47.429915 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:47.429926 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:47.430000 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:47.430103 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:46:47.430610 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:47.431850 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:47.434838 (+ 2988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:47.435750 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:47.435881 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:47.436830 (+ 949us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:46:47.437031 (+ 201us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:46:47.437108 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:47.437459 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 18:46:47.437555 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 18:46:47.444527 9698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:47.430714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37966 (local address 127.30.185.189:45943)
0811 18:46:47.430898 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:47.430905 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:47.437901 (+ 6996us) server_negotiation.cc:407] Connection header received
0811 18:46:47.438719 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:47.438726 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:47.438819 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:47.438905 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:46:47.439630 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:47.440547 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:47.442409 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:47.442910 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:47.443045 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:47.443589 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:46:47.443746 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:46:47.443836 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:47.444033 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 18:46:47.444130 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:46:47.448417 9721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:46:47.428849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53042 (local address 127.30.185.190:35703)
0811 18:46:47.441617 (+ 12768us) server_negotiation.cc:206] Beginning negotiation
0811 18:46:47.441625 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:46:47.441647 (+ 22us) server_negotiation.cc:407] Connection header received
0811 18:46:47.441761 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:46:47.441768 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:46:47.441981 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:46:47.442181 (+ 200us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:46:47.442961 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:47.444028 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:47.445916 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:46:47.446583 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:46:47.446723 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:46:47.447213 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:46:47.447720 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:46:47.447859 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 18:46:47.447983 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 18:46:47.448086 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12624,"thread_start_us":12508,"threads_started":1}
I20260811 18:46:47.458451 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 9014
I20260811 18:46:47.459801 9127 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:46:47.574290 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 9014
I20260811 18:46:47.607293 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 9147
I20260811 18:46:47.608736 9264 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:46:47.717553 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 9147
I20260811 18:46:47.751585 31462 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasky46AhN/build/asan/bin/kudu pid 9293
I20260811 18:46:47.752830 9418 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 18:46:47.863128 31462 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasky46AhN/build/asan/bin/kudu with pid 9293
2026-08-11T18:46:47Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5396 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (11035 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (397162 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST